NOTICE: This channel is no longer actively logged.
[00:14:26] *** Switeck has quit IRC [00:18:26] *** KyleK_ has quit IRC [00:27:50] *** thermal has joined #bittorrent [00:32:28] *** bittwist has quit IRC [00:34:28] *** burris has quit IRC [00:42:43] *** andar2 has quit IRC [00:51:32] *** rrr_ has quit IRC [00:59:09] *** rrr_ has joined #bittorrent [01:03:15] *** Andrius has quit IRC [01:05:12] *** ajaya has quit IRC [01:31:36] *** void^_ is now known as void^ [01:42:56] *** waldorf_ has quit IRC [01:45:12] *** init0_ has joined #bittorrent [01:46:34] *** HandheldPenguin is now known as HandheldPenguin` [01:46:45] *** waldorf_ has joined #bittorrent [01:50:43] *** bittwist has joined #bittorrent [01:56:54] *** init0 has quit IRC [01:58:22] *** tris has quit IRC [01:58:26] *** waldorf_ has quit IRC [02:31:50] *** BitTorrentBot has quit IRC [02:32:52] *** DeHackEd has quit IRC [02:32:53] *** stalled has quit IRC [02:33:47] *** stalled has joined #bittorrent [02:35:06] *** DeHackEd has joined #bittorrent [02:38:56] *** BitTorrentBot has joined #bittorrent [02:40:26] <DeHackEd> that bites [02:40:29] *** DeHackEd is now known as DHE [02:40:50] *** DeHackEd has joined #bittorrent [02:40:55] <DWKnight> connection fart? [02:41:13] <DeHackEd> linux OOM killer gone wild. It picked a good target though: X [02:41:54] <DeHackEd> And It Exploded. (tm) [02:44:59] <The_8472> what did it kill? [02:45:19] <DHE> X [02:45:26] <The_8472> hahaha [02:45:29] <DHE> which cascaded into the everything [02:45:53] <The_8472> how exactly did you manage to go OOM anyway? [02:46:11] <The_8472> i'm having trouble filling my physical memory these days... not to mention all the swap [02:46:18] <DHE> the C++ template compiler vs rtorrent [02:46:33] <DHE> 2 gigs of ram, 512M of swap, and -j2 just to make it doubly risky [02:47:14] <The_8472> well, i knew that C++'s templates were a crime, but i didn't know that they'd grab that much memory too [02:47:59] <DHE> crime? it's a damned felony [02:48:11] <The_8472> no, more like a crime agianst humanity... [02:48:40] <DHE> PID USER PR NI VIRT RES SHR S PU %MEM TIME+ COMMAND 3863 root 20 0 1733m 1.7g 2232 T 54 85.3 0:05.87 cc1plus [02:48:43] <DHE> whoops [02:48:53] <DHE> PID USER PR NI VIRT RES SHR S PU %MEM TIME+ COMMAND [02:48:55] <DHE> 3863 root 20 0 1733m 1.7g 2232 T 54 85.3 0:05.87 cc1plus [02:49:00] <Astro> The_8472: are you able to debug-fuly look something up on DHT? [02:49:12] <The_8472> hrm? [02:49:13] <DHE> not likely. DHT is 1.7 gigs of RAM [02:49:57] <TheSHAD0W> ... [02:49:59] <TheSHAD0W> Wha?? [02:50:07] <DHE> ./../rpc/command.h:178: internal compiler error: in type_unification_real, at cp/pt.c:9134 [02:50:07] <DHE> Please submit a full bug report, [02:50:07] <DHE> with preprocessed source if appropriate. [02:50:07] <DHE> See <URL:http://bugs.gentoo.org/> for instructions. [02:50:08] <DHE> awesome [02:50:41] <Astro> The_8472: (please don't kick me, it's CC content) http://torrents.thepiratebay.org/5117424/Nasty.Old.People.2009.XviD.5117424.TPB.torrent [02:51:10] <The_8472> what about it? [02:51:36] <Astro> try looking it up with router.bittorrent.com as entry [02:52:04] <The_8472> uhm. you should fill your routing table before you do any lookup [02:52:37] <Astro> implied [02:52:42] <The_8472> anyway, what kind of information should i gather from that lookup? [02:52:49] <Astro> get_peers? [02:52:56] <The_8472> so, the peer lists? [02:53:09] <Astro> after a find_node traverse, of course [02:53:27] <The_8472> yeeah, no. that's not how getpeers works [02:54:17] <Astro> while I re-read the BEP, could you try a lookup? [02:55:11] <The_8472> yes [02:55:48] * DHE tries compiling with -O1 and prays [02:59:30] <The_8472> i'm getting an empty peer list [03:03:18] <Astro> that may be a hint that the first prototype might be working :-) [03:04:23] <The_8472> i suggest that you post your results on forum.bittorrent.org once you're done [03:04:39] <Astro> now, ask again [03:04:41] <Astro> I turned them off [03:05:04] <Astro> yeah, I want to document it [03:08:00] <The_8472> hrrm, still an empty set. let's see if i'm actually debugging the right thing ^^ [03:09:36] *** HandheldPenguin` is now known as HandheldPenguin [03:13:01] * DHE successfully compiled rtorrent. that took longer than it should have [03:14:51] *** Gudy has joined #bittorrent [03:16:22] <The_8472> heh, ok. that was a bug on my end ^^ [03:20:35] <Astro> ok, does it work for you now? [03:21:14] <The_8472> [/85.17.143.185:6881, /192.87.224.194:50505, /85.17.143.185:6881, /200.27.115.18:6881, /84.216.44.4:18690, /41.214.64.194:12452, /70.51.229.57:21038, /76.204.200.204:52630, /89.54.173.213:57042, /201.37.20.147:88, /115.166.19.182:46861, /70.252.129.149:24388, /92.46.25.69:54482, /213.196.244.212:6688, /115.166.19.182:46861, /80.202.122.172:8000, /208.125.48.83:62721, /81.164.214.92:25068, /67.55.5.98:12839, /89.180.176.203:25707, /212.116.91.56:600 [03:21:20] <Astro> I've got to catch a bus in 10 minutes :) [03:21:23] <The_8472> err, ok. that was prettty long [03:22:00] <DHE> how do you exit rtorrent ? [03:22:02] <The_8472> http://pastebin.com/m395f146f <- peer list [03:22:11] <Astro> DHE: ^Q [03:22:12] <The_8472> so yes, it works [03:22:19] <The_8472> and my previous test was bogus [03:23:31] <Astro> retry now [03:24:50] *** gilles has joined #bittorrent [03:25:06] <The_8472> ok, sec [03:26:12] <Astro> unfortunately all those dht nodes on my machines are not self aware :) [03:27:07] <The_8472> yes, still getting a pretty long peer list [03:27:24] <Astro> hum, ok, I'll refine [03:27:26] <Astro> good nite [03:39:14] * DHE has no idea how to use DHT [03:40:14] <Nolar> no? you already share 2 our of 3 chars.... [03:40:48] <DHE> *thunk* [03:41:01] <Nolar> ;) [03:43:31] <DHE> oh, apparently rtorrent doesn't actually start torrents until you ask it to... [03:50:33] *** tris has joined #bittorrent [03:53:18] *** Miller` has joined #bittorrent [03:55:43] *** gilles has quit IRC [04:09:46] *** The_8472 has quit IRC [04:10:00] *** wadim has joined #bittorrent [04:10:02] *** wadim is now known as The_8472 [04:13:05] *** burris has joined #bittorrent [04:13:51] <alus> The_8472: what's with all the slashes [04:14:22] <The_8472> the data type i'm displaying there can store hostnames too [04:14:41] <The_8472> <hostname>/ip:port [04:15:14] <The_8472> but since they got instantiated directly with the ip it doesn't have a hostname stored [04:15:35] <alus> /ok man [04:15:43] <The_8472> ^^ [04:16:14] <The_8472> just using .toString() on a List<InetSocketAddress> [04:17:40] *** goussx has quit IRC [04:55:44] *** bt42 has joined #bittorrent [04:56:49] *** goussx has joined #bittorrent [05:00:19] <TheSHAD0W> Welp - my main credit card account is now useless. [05:00:35] <TheSHAD0W> Someone jacked it to use with Apple MobileMe. [05:03:04] *** bittwist has quit IRC [05:14:41] <bt42> :\ [05:14:48] <bt42> apple is hot for fraud these days [05:14:55] <bt42> used to be like dell business [05:23:50] *** HandheldPenguin is now known as HandheldPenguin` [05:34:22] <The_8472> <Astro> unfortunately all those dht nodes on my machines are not self aware :) <- he should talk about that with IBM [05:34:24] <The_8472> http://arstechnica.com/science/news/2009/11/ibm-makes-supercomputer-significantly-smarter-than-cat.ars [06:09:52] *** GTHK has quit IRC [06:53:54] *** stalled has quit IRC [06:54:18] *** stalled_ has joined #bittorrent [06:56:13] *** bittwist has joined #bittorrent [06:59:27] *** MassaRoddel has quit IRC [07:04:25] *** bt42 has quit IRC [07:23:24] *** MassaRoddel has joined #bittorrent [07:28:13] *** Snoopotic has quit IRC [07:39:30] *** KyleK_ has joined #bittorrent [08:07:00] *** KyleK_ has quit IRC [08:27:59] *** maik__ has joined #bittorrent [08:36:10] *** Andrius has joined #bittorrent [08:39:35] *** cyb2063 has joined #bittorrent [08:40:05] *** HandheldPenguin` is now known as HandheldPenguin [08:45:19] *** HandheldPenguin is now known as HandheldPenguin` [08:51:12] *** Andrius has joined #bittorrent [09:14:12] *** waldorf_ has joined #bittorrent [09:22:18] *** Andrius has quit IRC [09:23:29] *** Andrius has joined #bittorrent [09:27:55] *** waldorf_ has quit IRC [09:55:39] *** medecau has joined #bittorrent [09:59:06] *** goussx_ has joined #bittorrent [10:00:29] *** goussx has quit IRC [10:00:31] *** goussx_ is now known as goussx [10:31:21] *** waldorf_ has joined #bittorrent [11:19:11] *** burris has quit IRC [11:25:28] *** HandheldPenguin` is now known as HandheldPenguin [11:34:06] *** HandheldPenguin has left #bittorrent [11:51:53] *** goussx has quit IRC [12:09:18] *** medecau has left #bittorrent [12:57:52] *** uau has quit IRC [12:59:53] *** uau has joined #bittorrent [13:06:14] *** chelz has quit IRC [14:39:30] *** waldorf_ has quit IRC [15:36:45] *** waldorf_ has joined #bittorrent [16:22:03] *** fckgw has quit IRC [17:19:58] *** azimuth1 has joined #bittorrent [17:20:12] <azimuth1> hello guys [17:20:32] <azimuth1> has anyone experience on installing and running opentracker? [17:22:06] <The_8472> well, their homepage does state what you have to do to compile it, no? [17:22:25] <azimuth1> I have problems with running it [17:22:38] <azimuth1> basically it is up and running [17:22:57] <azimuth1> but when I try to download something it doesn't work [17:23:17] <azimuth1> even though clients are able to see that there are peers and seeds [17:23:28] <The_8472> well, what do you get as tracker error then? [17:24:04] <azimuth1> in statistics it says there were 2 HTTP 500 errors [17:24:29] <The_8472> i mean when the client tries to contact the server, what does it display as tracker status? [17:25:13] <azimuth1> hmm [17:25:21] <azimuth1> it just started to download [17:25:36] <azimuth1> as soon as I turned opentracker off [17:25:44] <azimuth1> but the status was OK [17:26:38] *** burris has joined #bittorrent [17:29:15] <cyb2063> [17:23:05CET] <azimuth1> but when I try to download something it doesn't work <- what does not work? [17:29:42] <azimuth1> well first the download didn't start [17:30:04] <azimuth1> even though scrape worked and client saw that there are other seeds and peers [17:30:43] <azimuth1> but then I put the torrent file to the opentracker's folder and restarted it and it started to download [17:31:07] <The_8472> uhm, things usually don't work that way [17:31:20] <azimuth1> yeah that confuses me [17:31:30] *** Gimoz has joined #bittorrent [17:31:38] <azimuth1> does opentracker need torrent files or some sort of llist of those? [17:31:50] <The_8472> only if you configure it in whitelist mode [17:31:54] <The_8472> otherwise it accepts any torrents [17:31:56] <azimuth1> coz on the website they say it has "-w mytorrents.list" key [17:32:09] <azimuth1> aha I see [17:32:52] <The_8472> if the clients see a seed/peer count that's a good indicator that the tracker is working [17:33:03] <cyb2063> opentracker only _tracks_ torrents, you need to publish the torrent files on your own. [17:33:21] *** Gimoz has quit IRC [17:34:03] <azimuth1> cyb2063 I know that [17:34:27] <azimuth1> I created a torrent file, put the tracker announce address in it and opened it on another computer [17:34:43] <The_8472> are you behind a router? [17:34:53] <cyb2063> well, did you configure opentracker in black/whitelist mode? [17:36:14] <azimuth1> cyb2063 no [17:36:36] <azimuth1> The_8472 I'm in a university network, so I guess yes [17:36:46] <The_8472> i mean behind a NATing router [17:37:20] <azimuth1> no [17:37:23] <cyb2063> well, in that case opentracker allows _all_ torrents that use its announce url. no need for -w listfile.txt [17:37:33] <azimuth1> all computer's have direct IP addresses [17:38:34] <The_8472> ok [17:38:36] <azimuth1> cyb2063, well I understand that but I opened the torrent file on another computer and it doesn't download... [17:39:02] <The_8472> it may take a while until the seed finds the peer or vice versa [17:39:15] <The_8472> it depends on which firewall allows incoming connections and all [17:40:29] <azimuth1> I'm trying to run opentracker in debug mode, thought maybe it will display some information, it doesn't :( [17:41:41] <azimuth1> I restarted seed and it started to download... [17:41:57] <azimuth1> it seems to be dependent on update time [17:42:39] <The_8472> well, then maybe one of the computers is firewalled [17:43:09] <The_8472> i.e. doesn't allow incoming connections [17:43:16] <azimuth1> I checked the seeding computer, seems to have client's port open [17:43:21] <The_8472> thus the non-firewalled one has to wait for the firewalled one to announce again and find the new IP [17:45:42] <azimuth1> it's working now [17:45:56] <azimuth1> thanks for tips anyway :) [17:47:38] *** KyleK_ has joined #bittorrent [17:57:31] *** azimuth1 has left #bittorrent [18:04:54] *** goussx has joined #bittorrent [18:21:46] *** gilles has joined #bittorrent [18:35:50] *** goussx has quit IRC [19:01:53] *** goussx has joined #bittorrent [19:24:14] *** KyleK_ has quit IRC [19:27:25] *** klapaucjusz has joined #bittorrent [19:27:48] <klapaucjusz> The_8472: are you still sending old nodes in reply to find_nodes, or have you fixed that? [19:27:57] <klapaucjusz> I'm seeing a lot of dead nodes in my routing table. [19:28:59] *** waldorf_ has quit IRC [19:29:20] *** KyleK_ has joined #bittorrent [19:31:55] <The_8472> klapaucjusz, i improved the stale-detection logic. but my timeouts are still very lenient [19:33:34] <The_8472> we have 3 states for nodes. good, questionable and bad. bad ones aren't included in node lists [19:33:47] <klapaucjusz> You should only be including good. [19:33:57] <klapaucjusz> What's the definition of good? [19:34:28] <The_8472> good = less than 2 failed queries and we heard of them within the last 15 minutes. [19:34:42] <The_8472> questionable = less than 2 failed queries [19:34:56] *** Snoopotic has joined #bittorrent [19:34:58] <klapaucjusz> You definitely shouldn't send questionable nodes. [19:35:01] <The_8472> bad = more than 8 failed queries or more than 2 failed queries + we haven't heard from them in the last 15 minutes [19:35:08] *** SuN has quit IRC [19:35:27] <klapaucjusz> If you do, questionable might never disappear from the table. [19:35:46] <klapaucjusz> Suppose a node N dies while in A's routing table. [19:35:47] <The_8472> they do [19:35:55] <klapaucjusz> A sends it to B -> N is questionable in B. [19:36:06] <klapaucjusz> A times it out, discards it. [19:36:19] <klapaucjusz> B sends it to A. [19:36:22] <The_8472> we never add things to the routing table we haven't seen ourselves [19:36:25] <klapaucjusz> Ad infinitum. [19:36:31] <klapaucjusz> Ah, okay, that changes things. [19:36:45] <klapaucjusz> Where do you put replies to find_nodes? [19:37:15] <The_8472> well, the node that replies means we have seen it, thus it may end up in the routing table [19:37:28] <klapaucjusz> (I'm still seeing a metric shitload of dead nodes going around.) [19:37:54] <klapaucjusz> Yeah, but where do you put nodes that you've learnt about but haven't contacted yet? [19:37:59] <The_8472> but the nodes/nodes6 lists returned by the reply only end up in a sorted set used for the query itself, it is discarded afterwards [19:38:11] <klapaucjusz> Please explain. [19:38:24] <The_8472> todo = new TreeSet<KBucketEntry>(new KBucketEntry.DistanceOrder(targetKey)); [19:38:45] <klapaucjusz> ...in English. [19:39:15] <klapaucjusz> How are you going to use them? [19:39:19] <The_8472> the lookup is an object. it contains a todo list. we dump the current routing table into the todo list and all nodes-reply lists too [19:39:34] <The_8472> and it's sorted by distance to the key, so we just have to poll nodes off that set to query them [19:40:26] <klapaucjusz> You're speaking of announces, right? [19:40:35] <klapaucjusz> I'm speaking of day-to-day maintenance. [19:40:37] <The_8472> of get_peers and find_node [19:41:12] <klapaucjusz> When do you send find_nodes? When a node replies for find_nodes, what do you do with the returned list of nodes? [19:41:20] <The_8472> well, day to day maintenance happens through PING and FIND_NODE messages mostly [19:42:09] <The_8472> we do FIND_NODE for a) bootstraps b) to fill buckets that are non-full once the bootstrap is done. [19:42:32] <The_8472> during a FIND_NODE lookup we use the same object with the same todo = new TreeSet<KBucketEntry>(new KBucketEntry.DistanceOrder(targetKey)); set [19:43:09] <The_8472> so the nodes/nodes6 lists don't go into the routing table directly, instead they get queried from the todo-set over the course of the FIND_NODE lookup [19:43:23] <The_8472> and if they then respond they may get inserted into the appropriate bucket or replacement bucket [19:43:44] <klapaucjusz> That's somewhat overkill, but yeah, it should work. [19:43:52] <klapaucjusz> How do you use the todo set? [19:44:04] <The_8472> also, any incoming packet qualifies to be inserted into the routing table [19:44:58] <The_8472> the todo set is sorted by the distance to the target key of the lookup. so we just dump all found node addresses into it [19:45:11] <The_8472> and then pull the currently closest ones from the set [19:45:16] <The_8472> for the next request [19:45:25] <The_8472> until the lookup meets its termination conditions [19:46:57] <The_8472> https://azsmrc.svn.sourceforge.net/svnroot/azsmrc/azsmrcplugins/trunk/lbms/plugins/mldht/kad/NodeLookup.java [19:47:13] <klapaucjusz> I don't understand. [19:47:42] <klapaucjusz> What lookup termination conditions are you speaking for? We're doing maintanance here, not peer announces. [19:47:49] <klapaucjusz> s/for/of/ [19:48:03] <Andrius> google should make codetranslate.google.com or something [19:48:11] <The_8472> if (todo.size() == 0 && getNumOutstandingRequests() == 0 [19:48:11] <The_8472> && !isFinished()) { [19:48:11] <The_8472> done(); [19:48:11] <The_8472> } else if (validReponsesSinceLastClosestSetModification >= DHTConstants.MAX_CONCURRENT_REQUESTS) { [19:48:11] <The_8472> done(); // quit after 15 nodes responses [19:48:11] <The_8472> } [19:48:21] <The_8472> those are the termination conditions for a node lookup [19:49:01] <klapaucjusz> Yeah, ok. [19:49:12] <klapaucjusz> Overkill, if you ask me, but should work. [19:49:19] <klapaucjusz> Then why am I seeing so many dead nodes? [19:50:08] <The_8472> are you sure they're dead? i have some cases with last-seen = 120 seconds and failed queries = 10 [19:50:17] <The_8472> it looks like they're not reachable but they can contact me [19:50:30] <The_8472> i suspect issues with teredo and nating [19:50:53] <klapaucjusz> I have four dead nodes in my bucket. [19:51:15] <The_8472> failed queries = 10 means they haven't been responding for a long while to our requests. but last seen = 120 means they contact us rather recently [19:51:22] <The_8472> so do i [19:51:37] <The_8472> but that's because we don't remove dead nodes unless we can replace them [19:51:40] <klapaucjusz> Ah, hold on. [19:52:12] <klapaucjusz> Node has sent you queries, but has never (yet) replied to a query of yours. [19:52:21] <klapaucjusz> Do you include such a node in find_nodes? [19:52:44] <The_8472> yes [19:52:46] <klapaucjusz> s/queries/requests/ [19:52:54] <klapaucjusz> Okay, that's the difference. I don't. [19:53:08] <klapaucjusz> Idea is -- if it's firewalled, we don't want to tell other nodes about it. [19:53:20] <klapaucjusz> Avoids table pollution. [19:53:25] *** thermal has quit IRC [19:53:35] <klapaucjusz> that's actually in BEP 5. [19:54:04] <klapaucjusz> Section Routing Table, second paragraph. [19:54:31] <The_8472> well, sending them in a nodes list shouldn't pollute the routing table [19:54:40] <klapaucjusz> ? [19:54:53] <klapaucjusz> I need a new node. I send find_nodes. You send me a bunch of nodes. [19:55:11] <klapaucjusz> I'll start pinging them, so you shouldn't send me unpingable nodes. [19:55:23] <klapaucjusz> No? [19:55:49] <The_8472> you shouldn't start pinging them. you should send them further find_node requests to perform a lookup towards your own ID [19:56:08] <klapaucjusz> The_8472: look, your implementation and mine are different. [19:56:19] *** thermal has joined #bittorrent [19:56:21] <MassaRoddel> lol [19:56:23] <The_8472> well, yeah. i'm following the extended kademlia specification [19:56:26] <klapaucjusz> But yes, it's most probably find_node that I'll send them. [19:56:38] <klapaucjusz> The point is: why are you sending me unreachable nodes? [19:56:41] <The_8472> the BEP5 is crud [19:56:51] <klapaucjusz> Whether I'll ping or find_nodes them, they're unreachable. [19:56:54] <The_8472> because i don't know whether they're unreachable or not [19:57:03] <The_8472> i'll stop sending them as soon as i know [19:57:07] <klapaucjusz> Then don't send them to me until you've found out. [19:57:31] <klapaucjusz> There's no point in both of us pinging them. Why don't you just do the job the once. [19:57:34] <The_8472> under stable conditions they get sorted out fast enough [19:58:02] <Andrius> I don't think you should assume stable conditions [19:58:05] <klapaucjusz> Under stable conditions stuff is easy, yeah. [19:58:11] <klapaucjusz> Stable conditions never happen. [19:58:21] * The_8472 points at the ipv4 table [19:58:22] <The_8472> they do [19:58:39] <klapaucjusz> 50% pollution in the IPv4 table. [19:58:41] <The_8472> i have nodes that are 2 days old in my ipv4 routing table [19:58:56] <klapaucjusz> Let's get back to the argument at hand. [19:59:08] <klapaucjusz> Could you please not send me nodes unless you've confirmed they are reachable? [19:59:38] <klapaucjusz> Either they are unreachable, in which case I don't want them. [19:59:50] <klapaucjusz> Or they are reachable, in which case you'll find out soon enough, and start sending them around. [20:00:27] <The_8472> considering that we don't do any active verification of reachability it could take up to 15 minutes until i might include them into responses, i'd rather not leave out valid nodes. especially not in the local buckets [20:00:50] <klapaucjusz> That's okay. I'll wait. What's 15 minutes between friends? [20:01:16] <The_8472> too much [20:01:25] <klapaucjusz> Sigh. [20:01:29] <klapaucjusz> The_8472: look. [20:01:41] <klapaucjusz> Unreachable nodes are being passed around. [20:01:43] <klapaucjusz> That's not good. [20:01:52] <The_8472> only by 1 hop [20:02:00] <The_8472> they're not propagated infinitely [20:02:16] <klapaucjusz> There are alternatives: you could just wait until you've pinged them. Or you could be so kind as to ping people in your local bucket more aggressively. [20:02:19] <The_8472> and you shouldn't trust anything in a nodes/nodes6 list [20:02:23] <The_8472> we certainly don't [20:02:26] <klapaucjusz> Of course I don't. [20:02:36] <The_8472> then i don't see the problem [20:02:41] <klapaucjusz> But I lose time and energy pinging nodes that my neighbours pass me. [20:02:54] <klapaucjusz> A good neighbour is one that doesn't pass information until it has verified it. [20:03:34] <klapaucjusz> As I've said, it doesn't cause major breakage, but it's useless, and there are alternatives. [20:03:45] <The_8472> well, the thing is if you leave nodes out then you might have an incomplete picture of your routing table. if you send too many all it does is a few packets more during a node lookup [20:03:59] <The_8472> so i rather err on the side of completeness [20:04:05] <klapaucjusz> Sigh. [20:04:11] <klapaucjusz> You're giving out false information. [20:04:15] <klapaucjusz> That's completeness? [20:04:20] <The_8472> no, i don't [20:04:33] <The_8472> i do give out potentially unverified information [20:04:34] * klapaucjusz gives up [20:05:31] <klapaucjusz> No, he doesn't. [20:05:37] <Andrius> lol [20:05:40] <The_8472> ? [20:06:08] <The_8472> you're talking behind my back? [20:06:19] <Andrius> yeah [20:06:21] <Andrius> I mean no [20:06:25] <The_8472> ... how nice [20:06:30] <MassaRoddel> klapaucjusz: also keep in mind that nodes do ban nodes therefore a node might not be reachable (anymore) from every other node [20:06:34] <Andrius> you should just agree to disagree or you'll end up arguing over this for the rest of the day [20:06:58] <klapaucjusz> MassaRoddel: so? [20:07:01] <The_8472> Andrius, i prefer reaching a mutual understand of each other's argument [20:07:39] <The_8472> *understanding [20:07:41] <Andrius> The_8472, we're not talking behind anyone's back. The "lol" was to klapaucjusz's "giving up" [20:08:06] <The_8472> hrrm, well, klapaucjusz was talking in 3rd person. [20:08:12] *** HandheldPenguin` has joined #bittorrent [20:08:14] *** HandheldPenguin` is now known as HandheldPenguin [20:08:20] <klapaucjusz> The_8472: yeah, about himself. Reach the two lines again. [20:08:28] <klapaucjusz> s/Reach/Read/ [20:08:32] <The_8472> heh, ok [20:08:35] <The_8472> you had me confused there [20:09:10] <klapaucjusz> I'm arguing that omitting non-confirmed nodes from find_nodes replies doesn't break anything. [20:09:11] <The_8472> our routing table management is very complicated and some behaviors are emergent of various other, simple algorithms [20:09:24] <The_8472> i sometimes have to rethink stuff myself to figure out how stuff actually works [20:09:32] <klapaucjusz> (Yeah, too complicated.) [20:09:43] <klapaucjusz> What if you omit an unconfirmed node in find_nodes? [20:09:54] <klapaucjusz> It'll just not get communicated around for at most 15 minutes. [20:10:10] <klapaucjusz> In practice, however, it probably asked *you* about your neighbours. [20:10:20] <klapaucjusz> So it'll contact me directly, and I can make my own decisions. [20:10:20] <The_8472> your local bucket may be incomplete and thus you might exclude newly-arrived nodes (churn) from responses [20:10:29] <klapaucjusz> That's okay. [20:10:52] <klapaucjusz> The newly-arrived node should normally contact me directly. (You gave him my address in reply to the find_nodes he sent you, right?) [20:11:10] <The_8472> only if you're one of my neighbors too [20:11:13] <klapaucjusz> At wich point I'll be able to make my own decisions about how to treat him. [20:11:28] <The_8472> otherwise it's rather unlikely that you're in my routing table and that i would send your data to him [20:11:42] <klapaucjusz> Except that among my 8 neighbours, there's 7 that are also neighbours of the newly-arrived node. [20:12:03] <klapaucjusz> Those 7 will send my address to the newly-arrived node. [20:12:25] <klapaucjusz> Er, not 7. Anywhere between 1 and 7. [20:12:51] <The_8472> what if i'm the 1? [20:12:53] <klapaucjusz> No, I was right. 7. [20:13:09] <The_8472> not if it's basically on the fringe of the key-range you're asking for [20:13:18] <klapaucjusz> Then the newly arrived node will contact *me* directly because you've given me my address in reply to find_nodes. [20:13:23] <klapaucjusz> We're going in circles. [20:13:53] <The_8472> well, ok. i'll put it another way. full buckets will be filtered and non-reachable nodes will be replaced rather fast [20:13:54] <klapaucjusz> If you disagree with this argument, there's another solution that might fit you better. [20:13:58] <The_8472> thus this mostly affects the local buckets [20:14:08] <The_8472> in which case it only applies to the terminal phase of any lookup [20:14:10] <klapaucjusz> When unconfirmed node arrives that's one of your neighbours, ping him eagerly. [20:14:16] <The_8472> where you'll get highly redundant information [20:14:20] * The_8472 does not see the harm [20:14:36] <The_8472> i'd rather include one too many than omitting one during that phase [20:14:36] <klapaucjusz> Because you're including unreachable nodes in *all* find_nodes queries. [20:14:43] <klapaucjusz> You're polluting *all* buckets. [20:14:56] <klapaucjusz> Not just your neighbouring bucket. [20:15:07] <klapaucjusz> That's a high cost to pay for what you call "completeness" [20:15:08] <The_8472> only if buckets are non-full [20:15:24] <The_8472> full buckets are unlikely to contain non-reachable nodes [20:15:26] <klapaucjusz> You slow down replacement. [20:15:45] <The_8472> hrm? [20:15:45] <klapaucjusz> Replacement has to ping all of the unreachable nodes you're pushing around before it find a good new node. [20:16:01] <The_8472> what is replacement? [20:16:41] <klapaucjusz> Replacing a stale node with a new one. [20:17:14] <The_8472> we have replacement buckets (as per the revised kademlia spec) for that. no find_node requests required for that [20:17:25] <klapaucjusz> So do I. [20:17:26] <The_8472> it's vastly more efficient that way [20:17:35] <klapaucjusz> But because of you, my replacement buckets are full of broken nodes. [20:17:41] <klapaucjusz> You're only pushing the issue further. [20:17:52] <klapaucjusz> So replacement is slowed down. [20:17:54] <The_8472> how can they contain broken nodes? [20:18:10] <klapaucjusz> Because they contain the broken nodes that *you* *sent* *me*. [20:18:26] <The_8472> then that's not a replacement bucket [20:18:31] <klapaucjusz> They shouldn't, but *you* are sending me *broken* *nodes*. [20:18:40] <The_8472> it's some cashed nodes-list responses. a replacement bucket is something else [20:18:45] <klapaucjusz> Huh? [20:19:21] <The_8472> http://infinite-source.de/az/whitepapers/kademlia_optimized.pdf chapter 4.1 [20:20:57] *** andar2 has joined #bittorrent [20:21:39] <The_8472> replacement buckets are not fed based on nodes lists from find_node queries. they're maintained passively based on incoming traffic [20:21:48] <The_8472> in fact, most of kademlia is maintained passively [20:22:11] <The_8472> find_node queries are only necesary to maintain _accuracy_ or in case you can't acquire enough nodes passively [20:22:21] <The_8472> and accuracy is what i'm aiming for [20:22:39] <klapaucjusz> (by publishing incorrect information) [20:22:52] <The_8472> *sigh* [20:23:08] <The_8472> by publishing potientially correct and potentially incorrect, i.e. unverified information [20:23:14] <The_8472> it's not incorrect a priori [20:24:16] <klapaucjusz> Yeah, I'm pretty much following 4.1 of the paper. [20:24:40] <klapaucjusz> Only difference: I do populate the replacement cache with both first-hand and second-hand information. [20:25:12] <klapaucjusz> The first-hand information can displace the second-hand information, so if enough first-hand information is available, there'll be no second-hand information. [20:25:22] <The_8472> excellent [20:25:36] <The_8472> we're using first hand information exclusively [20:25:56] <The_8472> since other nodes cannot be trusted [20:26:45] <klapaucjusz> The information from the replacement cache is not trusted. [20:26:52] <klapaucjusz> It'll be pinged before it's used. [20:27:05] <The_8472> that's another difference to chapter 4.1 then [20:27:09] <klapaucjusz> Your fault. [20:27:17] <klapaucjusz> Yep. [20:27:19] <The_8472> your deviation from the kademlia spec [20:27:21] <The_8472> not my fault ^^ [20:27:28] <klapaucjusz> Sue me. [20:27:34] <The_8472> ^^ [20:27:40] <The_8472> i'm just sticking to the spec [20:27:55] <The_8472> and for us it works pretty well [20:28:07] <The_8472> orders of magnitude better than before we used replacment buckets [20:29:12] * klapaucjusz is trying to find out where in the Kademlia paper (not spec) it says that sending wrong information in reply to find_nodes is a good idea [20:29:27] <The_8472> it doesn't i think [20:30:33] <klapaucjusz> But it does imply that your behaviour is the one. [20:30:38] <The_8472> but it puts emphasis on passive management, so we don't actively verify reachability unless we actually have to query that node. which can take quite a long time in the middle buckets [20:31:12] <The_8472> though tbh, i'm not sure if they even considered NAT issues [20:34:00] <The_8472> the thing is, we almost never do find-node queries on a healthy routing table. 1 bootstrap every 30 minutes and a few more after the bootstrap to verify/fill incomplete buckets [20:34:14] <The_8472> beyond that everything is as passive as possible [20:34:38] <The_8472> so costly find-node queries are not much of an issue [20:36:44] <The_8472> ... btw... that's exactly why i don't want teredo nodes if possible ^^ [20:43:07] <The_8472> hrrm... i lied. there is one more lookup every 10 minutes [20:43:27] <The_8472> but that's just for added frills. it feeds the DHT size estimator with new data [20:46:33] *** klapaucjusz has quit IRC [20:50:57] <The_8472> http://www.boingboing.net/2009/11/19/breaking-leaked-uk-g.html <- ... this must be a hoax or horribly misinformed... i hope [21:17:31] *** klapaucjusz has joined #bittorrent [21:18:09] <K`Tetch> i doubt its a hoax [21:18:16] <K`Tetch> we'l find out tomorow though [21:18:37] <K`Tetch> notice this, quietly changed - http://www.commonsleader.gov.uk/output/page2920.asp [21:22:51] <The_8472> funny how the emphasize the importance of digital communications infrastructure... and then propose disconnections [21:25:28] <The_8472> *they [21:27:59] *** Switeck has joined #bittorrent [21:36:16] <K`Tetch> thats mandy for you [21:36:30] <K`Tetch> now you understand why he's not run for election since, what, 2001? [21:57:11] *** andar has quit IRC [21:57:12] *** spoop has quit IRC [21:59:04] *** andar has joined #bittorrent [21:59:04] *** spoop has joined #bittorrent [22:00:02] *** spoop has quit IRC [22:02:42] *** spoop has joined #bittorrent [22:07:09] *** GTHK has joined #bittorrent [22:09:00] *** Gudy has quit IRC [22:11:55] <uriel> actually, beat me up, but I do like mandelson, he was a good EU trade representative, but on this he is totally, totally, totally, totally *fucked* [22:17:37] *** andar has quit IRC [22:18:13] *** KyleK_ has quit IRC [22:19:09] <Switeck> ok, who pressed the "nuke all" button? [22:19:25] * klapaucjusz hits uriel on the head with a stick. [22:21:15] *** Snoopotic has quit IRC [22:21:39] * Andrius sends a patch to all opensource dht implementations to hardcode uriel's IP in all responses [22:21:40] *** andar has joined #bittorrent [22:21:47] <The_8472> klapaucjusz, btw... which nodes are you instrumenting? [22:31:26] * klapaucjusz is impressed by Andrius [22:31:38] <klapaucjusz> The one on port 6970. [22:33:21] <The_8472> klapaucjusz, you should be instrumenting your bootstrap nodes btw [22:33:40] <The_8472> they get kicked from the routing tables, thus will see a lot less incoming traffic than normal nodes do [22:33:40] <klapaucjusz> The_8472: no, I shouldn't. [22:33:43] <klapaucjusz> Privacy issues [22:33:59] <The_8472> hrrm.. [22:34:18] <The_8472> i think privacy is actually less of an issue with them than with other nodes [22:34:30] <The_8472> since no/fewer get_peers requests shoudl be sent to them [22:35:36] <The_8472> i was more concerned about getting representative data [22:36:24] <klapaucjusz> At any rate, I'm not logging anything, and I'm only monitoring on one single node that I have log to a terminal. [22:36:39] <klapaucjusz> Monitoring the bootstrap nodes would probably require logging to a file, which is evil. [22:36:47] <klapaucjusz> Motto: don't be Google. [22:37:18] <The_8472> hehehe [22:38:29] <The_8472> i think i've read a great article somewhere titled (paraphrasing): sustainability in the information age - teaching the internet to forget [22:39:10] <The_8472> it was about how archiving everything, especially private things and conversations that used to be ephemeral can have real impact on the individual [22:39:23] <klapaucjusz> Yep. [22:39:25] <The_8472> like your new potential boss digging up your facebook profile and similar [22:39:31] <klapaucjusz> I'm not logging anything on IRC. [22:39:44] <klapaucjusz> Except for selected quotations that I copy manually using cut-n-paste. [22:40:07] <The_8472> this channel gets logged publicly btw [22:40:11] * The_8472 points at echelog [22:40:12] <klapaucjusz> You are not allowed to log anything I say. [22:40:26] *** nGTHK has joined #bittorrent [22:40:31] <klapaucjusz> echelog: you owe me $1000000 in moral damages. [22:40:36] <mpl> klapaucjusz: howdy. just wanted to let you know that you were right to push me on inspecting the bitfield. the bitfeld itself was fine but I was sending a length prefix which had a value -1 less than what it should be. it's all good now. :D [22:40:56] <klapaucjusz> mpl: glad to hear that. [22:41:08] <klapaucjusz> (You owe me a beer.) [22:41:15] <The_8472> there are three kinds of people in the world. those who do off-by-one errors and those who don't [22:41:15] <mpl> sure thing. [22:41:19] <klapaucjusz> (That's two beers.) [22:41:27] <mpl> The_8472: hush! [22:42:11] <The_8472> hey, the error is so common that it even got several names [22:48:01] *** GTHK has quit IRC [22:48:10] *** nGTHK is now known as GTHK [22:57:17] <klapaucjusz> The_8472: is there some torrent that we could download simultaneously, to test the IPv6 announcing? [22:57:48] <The_8472> how about just creating one [22:58:15] <klapaucjusz> Nah, too complicated for my little brain. [22:58:29] <klapaucjusz> ...or do as you wish. [22:58:30] <The_8472> then i'll do it ~~ [22:58:48] <klapaucjusz> Ok. [22:59:07] <klapaucjusz> Oh, please put a fake tracker entry. [22:59:24] <klapaucjusz> Transmission has a bug that prevents it from loading .torrent files with no trackers. [23:01:08] <The_8472> http://infinite-source.de/az/mldht_1.3.2.jar.torrent [23:01:54] <klapaucjusz> Nah, no tracker entry, breaks Transmission. [23:02:12] <The_8472> huh, it should have one. just not a http one [23:02:23] <klapaucjusz> Breaks transmission. [23:02:28] <klapaucjusz> :-( [23:02:35] <klapaucjusz> Okay, I'll edit it manually. [23:02:43] <The_8472> hrm, ok [23:02:55] <klapaucjusz> Ehm. [23:03:04] <klapaucjusz> No, it has a bunch of announce URLs. [23:03:08] <klapaucjusz> Transmission rejects it. [23:03:22] <The_8472> well, i tend to take things literally ^^ [23:03:42] <The_8472> and it's exactly 1 announce url [23:04:18] <The_8472> so, should i change it= [23:04:19] <The_8472> ? [23:04:28] <klapaucjusz> Ok. [23:04:44] <klapaucjusz> I'm downloading. [23:05:10] <klapaucjusz> Staring IPv6 DHT announce (poor, 34 nodes) [23:06:58] <The_8472> well, i did perform 8 stores, so it should be in the DHT if everything went right [23:07:18] <klapaucjusz> Learned 1 peers from DHT. [23:07:27] <klapaucjusz> (That's IPv4) [23:07:33] <Andrius> I wish I could set up IPv6 to send all kinds of malformed dht packets to you guys... [23:08:12] <The_8472> hrrrm [23:08:29] <The_8472> you got an ipv4 peer from the v6 dht? [23:08:40] <klapaucjusz> I got an IPv4 peer. [23:08:44] <klapaucjusz> I have no idea where from. [23:09:01] <The_8472> ah ok. i thought you might be running your node in ipv6-only mode or something like that [23:09:09] <klapaucjusz> I haven't gotten any IPv6 nodes yet. [23:09:13] <klapaucjusz> s/nodes/values/ [23:10:01] <klapaucjusz> Nope. [23:10:07] <klapaucjusz> Never got any IPv6 values. [23:10:28] <The_8472> hrrm, ok. let your node running [23:14:41] <The_8472> i found 2 IPv4 addresses from the IPv4 lookup and 0 IPv6 addreses from the IPv6 lookup [23:14:48] <The_8472> good news... no cross-contermination [23:14:54] <The_8472> bad news... something doesn't work right ^^ [23:14:55] <Nolar> any good (scalable) trackers out there besides tracker.openbittorrent.com ? [23:15:25] <The_8472> uh, the publicbt one, runs on opentracker too [23:15:35] <Nolar> which one? [23:15:39] <Nolar> also looking at /denis.stalker.h3q.com [23:15:56] <The_8472> http://tracker.publicbt.com/announce [23:16:01] <klapaucjusz> Okay. What's the info-hash? [23:16:16] <The_8472> 2265CEEF FCDC57AB 54B4B07F A9B0C92D 9E394B6F [23:16:22] <Nolar> context: ml .torrent get just passes info dict, not full torrent file, so we need to add a tracker [23:16:33] <klapaucjusz> I'm going to set up my peeking node near to it. [23:16:57] <The_8472> Nolar, there is another extension... tracker exchange. and trackers can be added to the magnet link too [23:17:32] <Nolar> The_8472 tracker exchange is live? [23:17:51] [23:18:00] <klapaucjusz> Okay. [23:18:10] <klapaucjusz> It's at 11 2c da 77 [23:18:11] <Nolar> The_8472 know enough to point parg in that direction? [23:18:23] <klapaucjusz> Let's wait a few minutes until the DHT reconverges. [23:18:38] <Andrius> Nolar, why do you "need" a tracker? [23:18:43] <The_8472> http://bittorrent.org/beps/bep_0028.html [23:19:55] <Nolar> Andrius to ensure we cross polinate mldht+azdht [23:20:02] <Nolar> The_8472 thanks [23:20:42] <Andrius> ok, makes sense [23:20:51] <klapaucjusz> Not to me. [23:21:07] <klapaucjusz> What problem does BEP 28 solve that's not solved by DHT+PEX? [23:21:18] <Nolar> in this case, if an az peer cant find the magnet info in the azdht, it can get the mldht data as a fallback [23:21:19] [23:21:42] <Nolar> bah [23:21:57] <Nolar> first time i've heard of it [23:22:13] <Nolar> but people throw up all sorts of crazy extensions there [23:22:50] <klapaucjusz> The_8472: can you please start an announce? [23:23:03] [23:23:16] <Nolar> heh [23:23:24] <The_8472> they don't seem to maintain one big changelog -.- [23:23:29] <Nolar> ya, would be good to rule this out upfront, before parg wasts too much time [23:24:04] <Nolar> alus you know? [23:24:44] <klapaucjusz> I was under the impression it's only supported by libtorrent(RB) [23:26:01] <Nolar> wouldnt surprise me [23:26:44] <klapaucjusz> The_8472: IPv6 DHT announce? Please? [23:26:51] <The_8472> ok, sec [23:27:24] <The_8472> if i'm doing something wrong then it's probably with the tokens [23:27:36] *** MassaRoddel has quit IRC [23:28:40] <klapaucjusz> No, I should at least be seeing myself. [23:28:53] <The_8472> the DHT is populated mostly by az nodes [23:29:00] <The_8472> so if there's some error with the storage... [23:29:09] <The_8472> ok, announcing [23:29:21] <The_8472> and done [23:30:20] <klapaucjusz> Done? [23:30:21] <klapaucjusz> You sure? [23:30:30] <The_8472> yes, sent 8 announces, 2 failed [23:30:48] <klapaucjusz> You haven't announced to 112cda77c2e572f4efe2ec28cc5e0cb6931315f9 [23:31:21] <The_8472> ok, then i'll debug that [23:33:10] *** cyb2063 has quit IRC [23:34:41] <klapaucjusz> Hold on, I'm changing id. [23:35:12] <The_8472> you have to be in someone's routingtable to end up in my result set [23:35:18] <The_8472> and you have to send a token of course [23:35:56] <klapaucjusz> Okay, I'm now 22653a6a8366f0bab7a3a1d37c6e1b3d7bb6e2d2 [23:36:05] <klapaucjusz> Yeah, let's wait for a few minutes. [23:36:08] <The_8472> you just missed the anounce -.- [23:36:13] <klapaucjusz> Right now, the DHT should mostly be a clique [23:36:22] <The_8472> http://pastebin.com/m7261c0de [23:36:45] <The_8472> the number of nodes already exceeds the number of routing table entries for me [23:37:34] <klapaucjusz> Hmm, yeah, you're right. [23:37:41] <klapaucjusz> There's some 100 nodes, and 32 table entries. [23:37:48] <The_8472> yes [23:38:00] <The_8472> that's how the IPv4 one looks like btw: http://pastebin.com/mc54e875 [23:38:03] <klapaucjusz> Okay, I'll tell you when I've converged [23:38:51] <klapaucjusz> (10 neighbours in 3 buckets) [23:39:13] <klapaucjusz> (12 neighbours in 4 buckets) [23:39:21] <The_8472> wow, that's slow [23:39:30] <The_8472> mine fills in ~20 seconds [23:39:50] <klapaucjusz> (Slowing down, I'm stuck on some dead nodes that for some reason my neighbours chose to tell me about.) [23:39:57] <The_8472> ^^ [23:40:34] <klapaucjusz> You're fe66? [23:40:38] <The_8472> yes [23:40:50] <klapaucjusz> Then it's fine, I just accepted your announce [23:41:01] <klapaucjusz> Now let's see if I can announce. [23:41:07] <The_8472> uhm [23:41:12] <The_8472> i haven't sent one right now oO [23:41:26] <klapaucjusz> 184 seconds ago [23:41:34] <The_8472> ah [23:41:38] <klapaucjusz> ...and counting [23:42:20] <klapaucjusz> Eek! [23:42:23] <The_8472> ah yes, there it is [23:42:23] <The_8472> sending announce to ID:22653A6A 8366F0BA B7A3A1D3 7C6E1B3D 7BB6E2D2 addr:/2001:0:d5c7:a2d6:23:42a2:a45a:216a:6970 [23:42:26] <klapaucjusz> There's 8 peers in the IPv4 DHT! [23:42:41] <klapaucjusz> Somebody has been following this discussion and trying to confuse us ;-) [23:42:50] <The_8472> kill them, kill them all! [23:43:10] <The_8472> or you're counting redundant responses [23:43:15] <klapaucjusz> No [23:43:49] <klapaucjusz> I got 1, 5, 6 responses. [23:44:06] <klapaucjusz> 7, 8, 2 [23:44:23] <klapaucjusz> And probably others, too lazy to scan my log. [23:45:10] <klapaucjusz> Okay. [23:45:15] <klapaucjusz> I've announced too, successfully. [23:45:26] <klapaucjusz> But for some reason I failed to get your address from the DHT. [23:47:06] <The_8472> same [23:47:43] <The_8472> what about yourself? can you get your own address? [23:47:49] <klapaucjusz> No. [23:48:12] <The_8472> hrrm, then i suspect something with storing keys in the DHT is wrong [23:48:23] <klapaucjusz> No, I can see the keys being stored. [23:48:39] <klapaucjusz> (My instrumented node can print out the set of values it stores.) [23:48:42] <The_8472> on your node. but maybe not on other nodes [23:49:00] <The_8472> but i have 25 stored keys too in my local database [23:49:03] <klapaucjusz> First hypothesis: the logging is broken ;-) [23:49:09] <The_8472> heh [23:50:15] <klapaucjusz> No, it ain't. [23:51:07] <The_8472> definitely storing ipv6 items in the database [23:52:30] *** JudgeSHAD0W has joined #bittorrent [23:53:05] <The_8472> so maybe the getpeers response is broken [23:53:06] *** A9[idle] has joined #bittorrent [23:54:55] <The_8472> hah, that's it indeed [23:55:18] <The_8472> i'm generating the peer list [23:55:20] <klapaucjusz> ? [23:55:28] <The_8472> but int inserting it into the message [23:55:43] <The_8472> but i'm not* [23:56:59] <klapaucjusz> and I'm even more stupid than you [23:57:15] <The_8472> i doubt that! [23:58:38] *** maik__ has quit IRC