tornadocontrib/go-ethereum
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,978 Commits 49 Branches 245 Tags 915 MiB
15f24ff189
Commit Graph

394 Commits

Author SHA1 Message Date
Anton Evangelatov
30263ad37d
swarm/storage: set false, only when we get a chunk back (#19599) 2019-05-31 11:13:34 +02:00
Elad
008d250e3c swarm/api/http: fix bzz-hash to return ens resolved hash directly (#19594) 2019-05-29 00:16:09 +02:00
Rafael Matias
cf38a3dc65 swarm/api: update mission statement (#19612) 2019-05-29 00:14:24 +02:00
Anton Evangelatov
509facd631
swarm/version: v0.4.1 unstable (#19587) 2019-05-17 10:14:39 +02:00
Anton Evangelatov
c3b317a4fc
swarm/version: v0.4.0 stable (#19586) 2019-05-17 10:12:03 +02:00
Anton Evangelatov
60386b3545 swarm/network: bump network id for 0.4 release (#19580) 
* swarm/network: bump network id for 0.4 release

* swarm/network: bump bzz protocol version

* swarm/docs: migration document v0.3 to v0.4

* swarm/storage/feed: gofmt lookup_test.go
 2019-05-16 18:29:12 +03:00
Javier Peletier
1e067202a2 swarm/feeds: Parallel feed lookups (#19414) 2019-05-16 15:47:11 +02:00
Anton Evangelatov
4b9c3bd39a
swarm/storage: disable open tracing on indices (#19578) 2019-05-15 15:06:57 +02:00
Anton Evangelatov
9b0d1b9ab2
swarm/metrics: collect metrics on datadir disk usage (#19576) 2019-05-15 12:22:32 +02:00
Elad
db83ba4067 cmd/swarm: skip export test on windows builds (#19555) 2019-05-13 10:27:27 +02:00
Anton Evangelatov
c8a77d8604
swarm/metrics: track runtime metrics (#19557) 2019-05-13 09:55:11 +02:00
holisticode
7744241886 swarm/network/stream: add pure retrieval test (#19552) 2019-05-11 12:55:06 +02:00
Elad
9b1543c282 swarm/network: update syncer metrics 2019-05-10 12:29:27 +02:00
Anton Evangelatov
297fa1855c swarm/network: measure addPeer and deletePeer to know if Kad rearranged 
swarm/storage: remove traces for put/get/set (#1389)

* swarm/storage: remove traces for put/get/set

* swarm/storage: remove Has traces
 2019-05-10 12:29:27 +02:00
Elad
84dfaea246 swarm: instrument setNextBatch 
swarm/storage/localstore: add gc metrics, disable flaky test
 2019-05-10 12:29:22 +02:00
Anton Evangelatov
3e9ba57669 swarm/storage: improve instrumentation 
swarm/storage/localstore: fix broken metric (#1373)

p2p/protocols: count different messages (#1374)

cmd/swarm: disable snapshot create test due to constant flakes (#1376)

swarm/network: remove redundant goroutine (#1377)
 2019-05-10 12:27:04 +02:00
lash
7f753461ca swarm/pss: disable failing handshake test 2019-05-10 12:26:59 +02:00
Anton Evangelatov
8802b9ce7f swarm-smoke: add syncDelay flag 
swarm/network: add want delay timer to syncing (#1367)

swarm/network: synchronise peer.close() (#1369)
 2019-05-10 12:26:55 +02:00
Elad
ad6c39012f swarm: push tags integration - request flow 
swarm/api: integrate tags to count chunks being split and stored
swarm/api/http: integrate tags in middleware for HTTP `POST` calls and assert chunks being calculated and counted correctly
swarm: remove deprecated and unused code, add swarm hash to DoneSplit signature, remove calls to the api client from the http package
 2019-05-10 12:26:52 +02:00
Janoš Guljaš
3030893a21 swarm/network: update syncing 2019-05-10 12:26:49 +02:00
Elad
12240baf61 swarm/chunk: add tags data type 
* swarm/chunk: add tags backend to chunk package
 2019-05-10 12:26:45 +02:00
Anton Evangelatov
f8eb8fe64c cmd/swarm-smoke: check if chunks are at most prox host 
swarm/network: measure how many chunks a node delivers (#1358)
 2019-05-10 12:26:41 +02:00
Elad
a1cd7e6e92 p2p/protocols, swarm/network: fix resource leak with p2p teardown 2019-05-10 12:26:37 +02:00
Janoš Guljaš
c1213bd00c swarm: LocalStore metrics 
* swarm/shed: remove metrics fields from DB struct

* swarm/schunk: add String methods to modes

* swarm/storage/localstore: add metrics and traces

* swarm/chunk: unknown modes without spaces in String methods

* swarm/storage/localstore: remove bin number from pull subscription metrics

* swarm/storage/localstore: add resetting time metrics and code improvements
 2019-05-10 12:26:33 +02:00
Anton Evangelatov
993b145f25 swarm/storage/localstore: fix export db.Put signature 
cmd/swarm/swarm-smoke: improve smoke tests (#1337)

swarm/network: remove dead code (#1339)

swarm/network: remove FetchStore and SyncChunkStore in favor of NetStore (#1342)
 2019-05-10 12:26:30 +02:00
Janoš Guljaš
996755c4a8 cmd/swarm, swarm: LocalStore storage integration 2019-05-10 12:26:26 +02:00
Janoš Guljaš
3873a7314d swarm/network: fix data races in TestInitialPeersMsg test (#19490) 
* swarm/network: fix data races in TestInitialPeersMsg test

* swarm/network: add Kademlia.Saturation method with lock

* swarm/network: add Hive.Peer method to safely retrieve a bzz peer

* swarm/network: remove duplicate comment

* p2p/testing: prevent goroutine leak in ProtocolTester

* swarm/network: fix data race in newBzzBaseTesterWithAddrs

* swarm/network: fix goroutone leaks in testInitialPeersMsg

* swarm/network: raise number of peer check attempts in testInitialPeersMsg

* swarm/network: use Hive.Peer in Hive.PeerInfo function

* swarm/network: reduce the scope of mutex lock in newBzzBaseTesterWithAddrs

* swarm/storage: disable TestCleanIndex with race detector
 2019-04-25 21:33:18 +02:00
gluk256
d9403690ec swarm/pss: Fix flaky TestProxNetwork (#19471) 2019-04-19 11:15:17 +02:00
Péter Szilágyi
78ec90717a
swarm/version: bump version due to Geth hotfix release 2019-04-17 15:50:41 +03:00
Javier Peletier
73fc65bb5b swarm/storage/feed: add context handling/cancellation to Swarm Feeds lookup, fix bad hint lookup bug (#19353) 
* swarm/storage/feed/lookup: Add context handling/forwarding

* swarm/storage/feed/lookup: Add test to catch bad hint

* swarm/storage/feed/lookup: Added context cancellation test
 2019-04-16 07:13:02 +02:00
Ferenc Szabo
26b50e3ebe cmd/swarm: fix resource leaks in tests (#19443) 
* swarm/api: fix file descriptor leak in NewTestSwarmServer

Swarm storage (localstore) was not closed. That resulted a
"too many open files" error if `TestClientUploadDownloadRawEncrypted`
was run with `-count 1000`.

* cmd/swarm: speed up StartNewNodes() by parallelization

Reduce cluster startup time from 13s to 7s.

* swarm/api: disable flaky TestClientUploadDownloadRawEncrypted with -race

* swarm/storage: disable flaky TestLDBStoreCollectGarbage (-race)

With race detection turned on the disabled cases often fail with:
"ldbstore_test.go:535: expected surplus chunk 150 to be missing, but got no error"

* cmd/swarm: fix process leak in TestACT and TestSwarmUp

Each test run we start 3 nodes, but we did not terminate them. So
those 3 nodes continued eating up 1.2GB (3.4GB with -race) after test
completion.

6b6c4d1c2754f8dd70172ab58d7ee33cf9058c7d changed how we start clusters
to speed up tests. The changeset merged together test cases
and introduced a global cluster. But "forgot" about termination.

Let's get rid of "global cluster" so we have a clear owner of
termination (some time sacrifice), while leaving subtests to use the
same cluster.
 2019-04-11 12:44:15 +02:00
Péter Szilágyi
b7dd225179
swarm/version: bump Swarm due to Geth hotfix release 2019-04-10 16:13:35 +03:00
Péter Szilágyi
5fc5971438
swarm/version: bump version due to Geth-only hotfix release 2019-04-09 12:19:24 +03:00
Péter Szilágyi
5b947c5004
swarm/version: bump version due to Geth maintenance release 2019-04-08 16:08:22 +03:00
Viktor Trón
0529015091
swarm/network: hive bug: needed shallow peers are not sent to nodes beyond connection's proximity order (#19326) 
* swarm/network: fix hive bug not sending shallow peers

-  hive bug: needed shallow peers were not sent to nodes beyond connection's proximity order
- add extensive protocol exchange tests for initial subPeersMsg-peersMsg exchange
- modify bzzProtocolTester to allow pregenerated overlay addresses

* swarm/network: attempt to fix hive persistance test

* swarm/network: fix TestHiveStatePersistance (#1320)

* swarm/network: remove trace lines from the hive persistance test

* address PR review comments

* swarm/network: address PR comments on TestInitialPeersMsg

 * eliminate *testing.T argument from bzz/hive protocoltesters
 * add sorting (only runs in test code) on peersMsg payload
 * add random (0 to MaxPeersPerPO) peers for each po
 * add extra peers closer to pivot than control
 2019-04-02 09:15:16 +02:00
lash
2f5b6cb442 swarm/network: Use different privatekey for bzz overlay in sim (#19313) 
* cmd/swarm, p2p, swarm: Enable ENR in binary/execadapter

* cmd/p2p/swarm: Remove comments + config.Enode nomarshal

* p2p/simulations: Remove superfluous error check

* p2p/simulation: Move init enode comment

* swarm, p2p/simulations, cmd/swarm: Use nodekey in binary record sign

* swarm/network, swarm/pss: Dervice bzzkey

* swarm/pss: Remove unused function

* swarm/network: Store swarm private key in simulation bucket

* swarm/pss: Shorten TextProxNetwork shortrunning test timeout

* swarm/pss: Increase prox test timeout

* swarm/pss: Increase timeout slightly on shortrunning proxtest

* swarm/network: Simplify bucket instantiation in servicectx func

* p2p/simulations: Tcpport -> udpport

* swarm/network, swarm/pss: Simplify + correct lock in servicefunc sim

* swarm/network: Cleanup after rebase on extract swarm enode new

* p2p/simulations, swarm/network: Make exec disc test pass

* swarm/network: Prune ye olde comment

* swarm/pss: Correct revised bzzkey method call

* swarm/network: Clarify comment about privatekey generation data

* swarm/pss: Fix syntax errors after rebase

* swarm/network: Rename misleadingly named method

(amend commit to trigger ci - attempt 5)
 2019-03-22 21:37:25 +01:00
gluk256
8d04154691 p2p/simulations: wait until all connections are recreated when uploading snapshot (#19312) 
* swarm/network/simulation: test cases refactored

* swarm/pss: minor refactoring

* swarm/simulation: UploadSnapshot updated

* swarm/network: style fix

* swarm/pss: bugfix
 2019-03-22 11:20:17 +01:00
lash
09924cbcaa cmd/swarm, p2p, swarm: Enable ENR in binary/execadapter (#19309) 
* cmd/swarm, p2p, swarm: Enable ENR in binary/execadapter

* cmd/p2p/swarm: Remove comments + config.Enode nomarshal

* p2p/simulations: Remove superfluous error check

* p2p/simulation: Move init enode comment

* swarm/api: Check error in config test

* swarm, p2p/simulations, cmd/swarm: Use nodekey in binary record sign

* cmd/swarm: Make nodekey available for swarm api config
 2019-03-22 05:55:47 +01:00
Anton Evangelatov
baded64d88
swarm/network: measure time of messages in priority queue (#19250) 2019-03-20 21:30:34 +01:00
gluk256
6e401792ce swarm/pss: negihbourhood addressing simulation tests (#19278) 
* swarm/pss: fixed bug in pss.process, test added

* swarm/pss: test case updated

* swarm/pss: WaitTillSnapshotRecreated() func added

* swarm/pss: snapshot test updated

* swarm/pss: WaitTillSnapshotLoaded() fixed

* swarm/pss: gofmt applied

* swarm/pss: refactoring, file renamed

* swarm/pss: input data fixed

* swarm/pss: race condition fixed

* swarm/pss: test timeout increased

* swarm/pss: eliminated the global variables

* swarm/pss: tests added

* swarm/pss: comments added

* swarm/pss: comment fixed

* swarm/pss: refactored according to review

* swarm/pss: style fix

* swarm/pss: increased timeout
 2019-03-16 08:39:38 +01:00
Janoš Guljaš
f180981273 swarm/shed: add vector uint64 field (#19279) 2019-03-15 15:12:45 +01:00
lash
4b4f03ca37 swarm, p2p: Prerequities for ENR replacing handshake (#19275) 
* swarm/api, swarm/network, p2p/simulations: Prerequisites for handshake remove

* swarm, p2p: Add full sim node configs for protocoltester

* swarm/network: Make stream package pass tests

* swarm/network: Extract peer and addr types out of protocol file

* p2p, swarm: Make p2p/protocols tests pass + rename types.go

* swarm/network: Deactivate ExecAdapter test until binary ENR prep

* swarm/api: Remove comments

* swarm/network: Uncomment bootnode record load
 2019-03-15 11:27:17 +01:00
Anton Evangelatov
1a3e25e4c1
swarm: tracing improvements (#19249) 2019-03-11 11:45:34 +01:00
Janoš Guljaš
9a58a9b91a swarm/storage/localstore: global batch write lock (#19245) 
* swarm/storage/localstore: most basic database

* swarm/storage/localstore: fix typos and comments

* swarm/shed: add uint64 field Dec and DecInBatch methods

* swarm/storage/localstore: decrement size counter on ModeRemoval update

* swarm/storage/localstore: unexport modeAccess and modeRemoval

* swarm/storage/localstore: add WithRetrievalCompositeIndex

* swarm/storage/localstore: add TestModeSyncing

* swarm/storage/localstore: fix test name

* swarm/storage/localstore: add TestModeUpload

* swarm/storage/localstore: add TestModeRequest

* swarm/storage/localstore: add TestModeSynced

* swarm/storage/localstore: add TestModeAccess

* swarm/storage/localstore: add TestModeRemoval

* swarm/storage/localstore: add mock store option for chunk data

* swarm/storage/localstore: add TestDB_pullIndex

* swarm/storage/localstore: add TestDB_gcIndex

* swarm/storage/localstore: change how batches are written

* swarm/storage/localstore: add updateOnAccess function

* swarm/storage/localhost: add DB.gcSize

* swarm/storage/localstore: update comments

* swarm/storage/localstore: add BenchmarkNew

* swarm/storage/localstore: add retrieval tests benchmarks

* swarm/storage/localstore: accessors redesign

* swarm/storage/localstore: add semaphore for updateGC goroutine

* swarm/storage/localstore: implement basic garbage collection

* swarm/storage/localstore: optimize collectGarbage

* swarm/storage/localstore: add more garbage collection tests cases

* swarm/shed, swarm/storage/localstore: rename IndexItem to Item

* swarm/shed: add Index.CountFrom

* swarm/storage/localstore: persist gcSize

* swarm/storage/localstore: remove composite retrieval index

* swarm/shed: IterateWithPrefix and IterateWithPrefixFrom Index functions

* swarm/storage/localstore: writeGCSize function with leveldb batch

* swarm/storage/localstore: unexport modeSetRemove

* swarm/storage/localstore: update writeGCSizeWorker comment

* swarm/storage/localstore: add triggerGarbageCollection function

* swarm/storage/localstore: call writeGCSize on DB Close

* swarm/storage/localstore: additional comment in writeGCSizeWorker

* swarm/storage/localstore: add MetricsPrefix option

* swarm/storage/localstore: fix a typo

* swamr/shed: only one Index Iterate function

* swarm/storage/localstore: use shed Iterate function

* swarm/shed: pass a new byte slice copy to index decode functions

* swarm/storage/localstore: implement feed subscriptions

* swarm/storage/localstore: add more subscriptions tests

* swarm/storage/localsore: add parallel upload test

* swarm/storage/localstore: use storage.MaxPO in subscription tests

* swarm/storage/localstore: subscription of addresses instead chunks

* swarm/storage/localstore: lock item address in collectGarbage iterator

* swarm/storage/localstore: fix TestSubscribePull to include MaxPO

* swarm/storage/localstore: improve subscriptions

* swarm/storage/localstore: add TestDB_SubscribePull_sinceAndUntil test

* swarm/storage/localstore: adjust pull sync tests

* swarm/storage/localstore: remove writeGCSizeDelay and use literal

* swarm/storage/localstore: adjust subscriptions tests delays and comments

* swarm/storage/localstore: add godoc package overview

* swarm/storage/localstore: fix a typo

* swarm/storage/localstore: update package overview

* swarm/storage/localstore: remove repeated index change

* swarm/storage/localstore: rename ChunkInfo to ChunkDescriptor

* swarm/storage/localstore: add comment in collectGarbageWorker

* swarm/storage/localstore: replace atomics with mutexes for gcSize and tests

* swarm/storage/localstore: protect addrs map in pull subs tests

* swarm/storage/localstore: protect slices in push subs test

* swarm/storage/localstore: protect chunks in TestModePutUpload_parallel

* swarm/storage/localstore: fix a race in TestDB_updateGCSem defers

* swarm/storage/localstore: remove parallel flag from tests

* swarm/storage/localstore: fix a race in testDB_collectGarbageWorker

* swarm/storage/localstore: remove unused code

* swarm/storage/localstore: add more context to pull sub log messages

* swarm/storage/localstore: BenchmarkPutUpload and global lock option

* swarm/storage/localstore: pre-generate chunks in BenchmarkPutUpload

* swarm/storage/localstore: correct useGlobalLock in collectGarbage

* swarm/storage/localstore: fix typos and update comments

* swarm/storage/localstore: update writeGCSize comment

* swarm/storage/localstore: global batch write lock

* swarm/storage/localstore: remove global lock option

* swarm/storage/localstore: simplify DB.Close
 2019-03-09 00:06:39 +01:00
Anton Evangelatov
bb55b0fb53 swarm/storage: add comparison towards leveldb.ErrNotFound (#19243) 
* swarm/storage: add comparison towards leveldb.ErrNotFound

* swarm/storage: wrap leveldb ErrNotFound
 2019-03-08 17:28:57 +01:00
Anton Evangelatov
2cfe0bed9f swarm: fix relationship between spans in open tracing (#19236) 
* swarm/network: propagate span with ctx

* swarm/network: try to stop stream.send.request spans on time

* swarm/storage: add chunk ref as a log to netstore.fetcher span
 2019-03-08 08:52:25 +01:00
Janoš Guljaš
a6e5c6a2cc swarm/storage/localstore: fix synchronization in TestDB_gcSize (#19235) 2019-03-08 05:59:59 +01:00
Janoš Guljaš
eb199f1fc2 swarm: localstore hasser (#19230) 2019-03-07 10:07:54 +01:00
Ferenc Szabo
d45f8d1880 swarm/network: remove *WithServer tests from stream package (#19223) 
These tests never run as the build tag excluded them from the CI
execution. As a results the (dead) code got out of sync with other
parts of Swarm and now they would not even compile. => Removed.

resolves ethersphere/go-ethereum#1238
 2019-03-07 09:27:56 +01:00
holisticode
a87776a5fe swarm/network/stream: Fix flaky tests in GetSubscriptionsRPC test (#19227) 
* swarm/network/stream: fixed timing issues

* swarm/network/stream: only count first iteration of subscriptions

* swarm/network/stream/: fix linter errors
 2019-03-07 09:24:28 +01:00