[00:03:17] *** bryan986 has joined #bittorrent
[00:39:36] *** thirdwheel has joined #bittorrent
[00:40:11] <thirdwheel> hey all, do BT clients usually keep dedicated lines for upload and download, or is it common for upstream and downstream to share a channel between leechers?
[00:40:33] <Nolar> latter
[00:40:44] <thirdwheel> thanks, that helps a lot
[00:40:49] <Nolar> bt connections are bi-directional
[00:41:52] <thirdwheel> i'm using an openbsd firewall to limit my bt downloads during n-peak time, and that explains the reason why my bt queue is full and spilling over into the normal queue
[00:51:46] *** thirdwheel has left #bittorrent
[01:12:43] *** TheSHAD0W has quit IRC
[01:18:09] *** ryanprior has joined #bittorrent
[01:37:32] *** andar2 has quit IRC
[02:05:08] *** ajaya has joined #bittorrent
[03:10:33] *** The_8472 has quit IRC
[03:30:50] *** kwinz2 has quit IRC
[03:31:14] *** kwinz2 has joined #bittorrent
[03:33:28] *** ajaya has quit IRC
[04:25:31] *** init0_ has joined #bittorrent
[04:38:26] *** init0 has quit IRC
[04:38:37] *** kwinz2 has quit IRC
[04:41:12] *** kwinz2 has joined #bittorrent
[04:44:43] *** TheSHAD0W has joined #bittorrent
[04:46:14] *** goussx has quit IRC
[05:12:37] *** burris has quit IRC
[05:13:49] *** burris has joined #bittorrent
[05:17:20] *** goussx has joined #bittorrent
[05:23:23] *** Miller` has joined #bittorrent
[05:28:52] *** firefly2442 has joined #bittorrent
[05:30:20] <firefly2442> Sort of OT but.... can anyone point me in the direction of a protocol or service for network autodiscovery on a LAN?  essentially how it works.
[05:30:54] <firefly2442> If I have two machines on a LAN and I don't know the IP address of the machine, can I send out packets or use a protocol to see if that machine is on and active?
[05:32:13] <Miller`> firefly2442: you can use broadcast udp packets, or if you want something pre-packaged, look at DNS-SD and Bonjour
[05:32:38] <TheSHAD0W> There are two autodiscovery services...
[05:32:42] <TheSHAD0W> UPnP and...
[05:32:46] <TheSHAD0W> What's that other one.
[05:32:56] <Miller`> oh, i totally read that as you want service auto-discovery
[05:33:15] <Miller`> ignore what i just said
[05:33:19] <TheSHAD0W> Zeroconf.
[05:33:30] <TheSHAD0W> There's also Bonjour.
[05:33:56] <TheSHAD0W> Zeroconf is pretty popular.
[05:33:59] <TheSHAD0W> Oh.
[05:34:05] <TheSHAD0W> Bonjour is Apple's version of Zeroconf.
[05:35:15] <firefly2442> so the easiest way if I just wanted to program something simple would be to use UDP?  broadcast is a flag set before the packet is sent?
[05:36:42] * firefly2442 reads a little more.. ahh OK, I think I get it
[05:37:09] <firefly2442> thanks
[05:48:29] *** bryan986 has quit IRC
[06:05:21] *** firefly2442 has quit IRC
[06:33:28] *** MassaRoddel has quit IRC
[06:43:39] *** kwinz2 has quit IRC
[06:47:12] *** kwinz2 has joined #bittorrent
[07:16:24] <TheSHAD0W> http://markproffitt.com/2009/04/29/larry-lessigs-video-about-copyright-abuse-is-abused-using-dmca/ - in case you haven't seen this video...
[07:18:59] <Miller`> ...you've gotta be kidding..............wow
[07:19:18] *** _rafi_ has joined #bittorrent
[07:31:58] <TheSHAD0W> http://www.spacedaily.com/reports/T_Pyxidis_Soon_To_Be_A_Type_Ia_Supernova_999.html - and the whole global warming argument is now moot.
[07:34:59] *** kwinz2 has quit IRC
[07:38:13] *** kwinz2 has joined #bittorrent
[07:40:31] *** KyleK_ has joined #bittorrent
[08:15:12] *** KyleK_ has quit IRC
[08:29:34] *** kwinz2 has quit IRC
[08:33:45] *** kwinz2 has joined #bittorrent
[08:38:27] *** MassaRoddel has joined #bittorrent
[08:39:40] *** chelz has joined #bittorrent
[08:54:43] *** goussx_ has joined #bittorrent
[09:10:47] *** GTHK has quit IRC
[09:10:49] *** goussx has quit IRC
[09:10:50] *** goussx_ is now known as goussx
[09:22:55] *** chelz has quit IRC
[09:27:53] *** rrr_ has quit IRC
[09:44:32] *** swin has joined #bittorrent
[09:44:33] *** swinokur has quit IRC
[09:44:36] *** swin is now known as swinokur
[10:07:21] *** Waldorf has joined #bittorrent
[10:29:04] *** rrr_ has joined #bittorrent
[11:26:27] *** ryanprior has quit IRC
[11:41:07] *** The_8472 has joined #bittorrent
[12:42:36] *** cyb2063 has joined #bittorrent
[16:20:24] *** Gottaname has quit IRC
[16:22:48] *** Gottaname has joined #bittorrent
[16:28:25] *** Gottaname has quit IRC
[17:06:17] *** Gottaname has joined #bittorrent
[17:45:56] *** goussx has quit IRC
[18:01:12] *** klapaucjusz has joined #bittorrent
[18:02:25] <klapaucjusz> Any forum.bittorrent.org moderators here?
[18:02:57] <klapaucjusz> My post of 2010/01/01 has disappeared, with no indication why.
[18:03:08] <klapaucjusz> Deliberate moderation, or a glitch?
[18:03:12] <klapaucjusz> Shall I submit it again?
[18:03:59] <alus> klapaucjusz: a glitch, I'm sure
[18:04:42] <klapaucjusz> Hi alus.  It was about http://www.pps.jussieu.fr/~jch/software/bittorrent/bep-dht-minor-extensions.html
[18:04:49] <klapaucjusz> which I'd like to hear your opinion about.
[18:06:09] <The_8472> yeah, i wondered too. i have moderator rights but i don't see it anywhere
[18:06:21] <alus> hm, these seem pretty straightforward and good.
[18:06:25] <The_8472> btw, is there a decent editor for RST?
[18:06:29] <klapaucjusz> Emacs.
[18:06:37] <The_8472> i said decent
[18:06:46] <The_8472> which means WYSIWYG
[18:06:49] <alus> is there really a reason to use 1 and 2 for the drop values? why not strings?
[18:06:53] <klapaucjusz> alus: the one bit that might be controversial is the name and syntax of the "drop" parameter.
[18:07:02] <klapaucjusz> :-)
[18:07:07] <alus> :)
[18:07:11] <klapaucjusz> Suggestions?
[18:08:03] <alus> well, drop: overloaded and drop: bootstrap seem to be better. I'm not worried about the extra bytes but maybe "o" and "b" would be fine
[18:08:46] <klapaucjusz> You're happy with the name "drop" ?
[18:08:52] <The_8472> ovrld, btstrp ^^
[18:09:14] <klapaucjusz> The_8472 is learning Serbian?
[18:09:26] <alus> "drop" does indicate a bit of what's going on. I can't think of anything better
[18:09:59] <klapaucjusz> Further issue: what shall be the behaviour of a node that receives a "drop" parameter with a value that it doesn't grok?
[18:10:02] <The_8472> no, just dropping vowels
[18:10:12] <alus> I was wondering that.
[18:10:16] <klapaucjusz> Two possibilities: ignore, treat as overload.
[18:10:23] <alus> seems safest to ignore it
[18:10:26] <klapaucjusz> Ok.
[18:11:05] <The_8472> btw, have you assessed behavior modifications of clients that wish to be dropped instead of explicit values?
[18:11:07] <klapaucjusz> The_8472: do you want to be credited nominally in the acknowledgments, or is "The 8472" fine?
[18:11:18] <The_8472> The 8472 is fine
[18:11:20] <klapaucjusz> ?
[18:12:25] <alus> yeah, implementing this will require us to ignore our DHT rate limiter
[18:12:38] <alus> hm.
[18:12:48] <alus> or, maybe not
[18:12:50] <The_8472> well... clients can simply adjust their behavior if they want to be dropped from others' routing tables, e.g. by not responding to queries. or in the case of bootstrap nodes to returning a node ID with a) the 1st bit inverted (that way they'll never end up in the local bucket) b) not responding to any queries but lookups targeting close to the asking node
[18:12:55] <klapaucjusz> The short-term work is to implement it on reception only.
[18:13:35] <klapaucjusz> In Transmission, I intend to have two thresholds in the rate limiter.
[18:13:48] <klapaucjusz> Mark for small values, drop for larger values.
[18:14:14] <The_8472> can you describe what you're trying to fix?
[18:14:26] <klapaucjusz> This is similar with what modern RED implemntations do in the presence of ECN (except that ECN availability is explicitly signalled, so the tradeoffs are somewhat different.)
[18:14:54] <klapaucjusz> The_8472: okay, will do.
[18:16:44] <The_8472> because using drop under the wrong conditions could just make things worse
[18:16:46] <klapaucjusz> Updated.
[18:17:15] <klapaucjusz> (Haven't added the rationale yet.)
[18:17:35] *** goussx has joined #bittorrent
[18:17:45] <The_8472> nonono
[18:17:52] <The_8472> i want you to explain what the issue is
[18:17:59] <The_8472> why you came up with the BEP
[18:18:06] <klapaucjusz> That's what I mean with "I haven't added the rationale yet."
[18:18:29] <The_8472> adding it there is pointless if your rationale turns out to be wrong :P
[18:18:46] <klapaucjusz> Ok.
[18:19:04] <klapaucjusz> Over time, a stable DHT node becomes increasingly popular.
[18:19:35] <klapaucjusz> As the number of other DHT nodes that route through it grows without bound.
[18:19:48] <klapaucjusz> There are two known mechanisms for counter-balancing that effect:
[18:19:59] <klapaucjusz> rate-limiting requests, and dropping, and changing node-ids.
[18:20:05] <klapaucjusz> Both have negative side-effects.
[18:20:53] <The_8472> ok, but that is more or less orthogonal to the 'unwillingness to store' aspect.
[18:21:14] <klapaucjusz> It's completely orthogonal, of course.
[18:21:28] <klapaucjusz> I was assuming that much is clear.
[18:21:43] <The_8472> distinguishing between those cases is _very_ important.
[18:21:44] <klapaucjusz> I'll make it clearer.
[18:24:12] <klapaucjusz> Added "To do" notes.
[18:24:21] <The_8472> mark/drop might start sending drop requests for the wrong reasons when you're actually seeing lots of traffic because you're responsible for a popular key for example
[18:28:05] <klapaucjusz> The_8472: fancy writing the paragraph that explains what widening is?
[18:28:32] <The_8472> can do. can you make a new thread so i can post it there?
[18:29:16] <klapaucjusz> http://forum.bittorrent.org/viewtopic.php?id=148
[18:32:40] *** medecau has joined #bittorrent
[18:51:07] <The_8472> hrrm, the logic for correct and efficient perimeter widening is actually not quite simple in the case N > K ~~
[18:51:22] <The_8472> i basically have to explain my implementation ^^
[19:02:52] <klapaucjusz> The_8472: don't.
[19:03:12] <klapaucjusz> Please think it over, you'll find a descriptive way to explain.
[19:03:33] <klapaucjusz> At worst, just provide a reference.
[19:03:53] <The_8472> hrmhrm, ok
[19:07:30] <The_8472> klapaucjusz: http://forum.bittorrent.org/viewtopic.php?pid=900#p900
[19:07:33] <The_8472> simple enough?
[19:10:11] *** GTHK has joined #bittorrent
[19:10:59] <The_8472> sec, reformatting it
[19:11:07] <The_8472> it's... still difficult to explain
[19:11:08] <klapaucjusz> Nope, way too many details.
[19:11:16] <The_8472> ~~
[19:11:38] <klapaucjusz> You need to carefully think what is essential for interoperability, and what is just an implementation detail.
[19:11:53] <The_8472> the problem is if you get it wrong it either still puts load on the nodes which want to avoid load or you store values too far away from the key unnecessarily, which is already happening
[19:12:27] <klapaucjusz> Think declaratively, not imperatively.
[19:12:37] <klapaucjusz> Describe where the key should find itself, not how to compute the key.
[19:12:40] <klapaucjusz> s/key/location/
[19:12:50] <The_8472> ok
[19:13:12] <klapaucjusz> And please don't edit your message, put another one.
[19:13:16] <klapaucjusz> So we can compare.
[19:15:30] <klapaucjusz> Hold on, I'll suggest a wording on the forum.
[19:18:45] <klapaucjusz> Done.
[19:21:57] <The_8472> nono ^^
[19:22:43] <The_8472> if you encounter a node that doesn't send a token but has values it should mean the key is very popular and overloaded
[19:22:53] <The_8472> you don't want to continue the lookup
[19:23:01] <The_8472> since then you'd just contribute to the load
[19:23:34] <The_8472> it's important that the whole thing terminates as early as possible without terminating too far from the target
[19:26:48] * klapaucjusz is pondering
[19:27:59] <The_8472> my implementation currently errs on the too-far side, so i already toned down the requirements a bit in the forum post
[19:33:50] *** ajaya has joined #bittorrent
[19:36:48] *** andar2 has joined #bittorrent
[19:43:26] *** kwinz2 has quit IRC
[19:47:17] <klapaucjusz> The_8472: that's a lot of help.
[19:47:25] <klapaucjusz> I'll ping you when I'm happy with our prose.
[19:47:41] * klapaucjusz should have been a jeweller.
[19:48:06] <The_8472> polishing and polishing ^^
[19:50:55] <klapaucjusz> Is "Network throughput" comprehensible for non-specialists?  I'm trying to avoid using the term "bandwidth", which doesn't mean what most people think it means.
[19:51:28] <The_8472> traffic?
[19:51:38] <andar2> rate?
[19:51:39] <The_8472> capacity?
[19:51:43] <klapaucjusz> "dedicate more network throughput"
[19:52:05] <klapaucjusz> "capacity" is good.
[19:52:39] <The_8472> anyway, has this been a real problem?
[19:53:04] <klapaucjusz> For bootstrap nodes, it certainly is.
[19:53:40] <The_8472> bootstrap nodes can do behavioral adjustments though, like flipping IPs
[19:53:45] <The_8472> i mean for regular nodes
[19:54:07] <klapaucjusz> I've had to raise my token bucket parameters from what I initially envisioned.
[19:54:48] <The_8472> because the wrong behavior in a widespread implementation could have serious impact on the DHT. worst case scenario would be changing O(log(n)) to O(n) hops. n = 4M-5M ...
[19:54:53] <klapaucjusz> http://bittorrent.pastebin.com/m1d28c9ca
[19:55:36] <The_8472> looks good
[19:55:51] <The_8472> and for a popular key not just the storage requests can be a burden
[19:56:08] <The_8472> the get_peers traffic alone, without announces could be too much
[19:56:22] <The_8472> though i guess we'd have to test that
[19:56:45] <klapaucjusz> You may be overreacting ;-)
[19:56:45] <The_8472> pick the most popular torrent we can find and then insert a node near the responsible ID
[19:57:13] <The_8472> i just said we have to test it :P
[20:00:16] <The_8472> if actually just the store requests are the issue and not the GET_PEERS traffic then it makes things a lot simpler
[20:00:37] <The_8472> wrt to perimeter widening
[20:00:44] <The_8472> i was trying to cover both issues
[20:02:25] * klapaucjusz ponders again
[20:03:51] <klapaucjusz> How do you spell "signalling"
[20:03:52] <klapaucjusz> ?
[20:04:07] <klapaucjusz> My dictionary disagrees with me.
[20:05:12] <The_8472> i agree with your dictionary
[20:05:53] <kjetilho> "n American English, in a multisyllabic word with a final consonant directly preceded by a single vowel, that consonant does not get doubled if the stress does not fall on the last syllable."
[20:06:11] <kjetilho> so it's a British vs. American thing
[20:07:01] <klapaucjusz> Damn yanks.
[20:07:02] <kjetilho> (in British, 'l' is *always* doubled)
[20:07:07] <klapaucjusz> They managed to sell me a dictionary.
[20:08:29] <The_8472> http://forum.bittorrent.org/viewtopic.php?pid=905#p905
[20:09:47] <klapaucjusz> Understood.  So there are three different approaches, from the dumbest to the smartest.
[20:09:55] <klapaucjusz> 1. use 7 instead of 8.
[20:10:00] <klapaucjusz> 2. widen naively.
[20:10:03] <klapaucjusz> 3. widen smartly.
[20:10:22] <klapaucjusz> I think that justifies a separate implementation note.
[20:10:30] * klapaucjusz is going to make more coffee.
[20:12:03] * klapaucjusz wishes that smart people like The_8472 could finally learn to write, rather than relying on dumb people like himself to do that job.
[20:12:19] <The_8472> btw, how have you implemented yours? do you announce on every get-peers lookup or do you distinguish between get-peers-only lookups and announces?
[20:12:37] <klapaucjusz> ?
[20:13:14] <The_8472> well. you can terminate get_peers lookups more aggressively when you don't have to find nodes to announce to
[20:13:33] <The_8472> so the lookups can be used for 2 things
[20:14:12] <The_8472> either just get a handful of peers and be done with it or meticulously look for the K closest nodes where you announce to
[20:16:51] <klapaucjusz> I see.
[20:16:58] <klapaucjusz> No, we always announce in Transmission.
[20:17:17] <klapaucjusz> The DHT library has an interface to perform searches without announcing, but the lookup algorithm remains the same.
[20:17:57] <klapaucjusz> In either case, we pass values obtained from get_peers to the transmission core as soon as we receive them, i.e. without waiting to determine the 8 closest nodes.
[20:18:08] <The_8472> yeah, same here. i was just wondering if it woudl worth distinguishing those 2 cases
[20:18:30] <The_8472> i collect all values and filter them for uniqueness first
[20:18:52] <klapaucjusz> That happens in the Transmission core, and is done on the fly.
[20:18:58] <The_8472> ah, nice
[20:19:31] <klapaucjusz> It's pretty nice to be able to get peers before the search has converged, since it usually gives you a few peers right away.
[20:19:37] <The_8472> i think it happens somewhere in Az too, but we do additional statistics on things, so returing one set is better than returning many small ones
[20:19:53] <klapaucjusz> ...at which point PEX starts doing magic.
[20:20:22] <klapaucjusz> You could still filter on the fly, i.e. by passing new peers to the core straight away, but keeping a list to determine uniqueness.
[20:20:22] <The_8472> hrrm, well. we wait with the pexing
[20:20:35] <The_8472> you don't want pex-lists right away after connecting to peers
[20:20:45] <klapaucjusz> ?
[20:20:46] <The_8472> that woudl be a serious waste of bandwidth
[20:21:02] <klapaucjusz> ?
[20:21:18] <The_8472> seed connects to seed, you're going to disconnect him soon/immediately. sending pex would just be wasting bandwidth on a useless connection
[20:21:32] <klapaucjusz> How much is a 50 element list for IPv4?  300 bytes?
[20:21:52] <The_8472> + flags
[20:22:01] <The_8472> so maybe 400
[20:22:28] <klapaucjusz> When seed meets seed, they should share lists of leechers before disconnecting, no?
[20:23:00] <klapaucjusz> Especially if one of the two is unreachable.
[20:23:27] <The_8472> we only consider pex lists of currently connected seeds. so if the other seed is disconnected so are their pex lists
[20:23:44] <klapaucjusz> Please read again.
[20:24:12] <The_8472> i know what you mean
[20:24:18] <The_8472> i was just talking about our implementation
[20:24:35] * klapaucjusz goes back to writing.
[20:24:36] <The_8472> on disconnecting all values obtained through pex are invalidated
[20:24:50] * klapaucjusz claims that's stupid.
[20:25:44] <The_8472> our pex works by maintaining an exhaustive live view of what everyone we're connected to is connected to
[20:26:13] <The_8472> which means all pex entries are fresh
[20:26:25] <klapaucjusz> I know, you've explained that to me already.
[20:26:55] <klapaucjusz> I still claim it's suboptimal.
[20:27:24] <klapaucjusz> (None of the above should be taken as meaning that Transmission's handling of PEX is optimal, although I believe that the implementation in 1.80 is pretty good.)
[20:28:07] *** kwinz2 has joined #bittorrent
[20:28:33] <klapaucjusz> (Partly thanks to your explanations, by the way.)
[20:28:54] <klapaucjusz> (And to my sheer genious, of course ;-)
[20:31:19] <The_8472> http://forum.bittorrent.org/viewtopic.php?pid=906#p906
[20:31:41] <The_8472> hehe
[20:32:15] <The_8472> but yes, it might be slightly suboptimal in the startup case
[20:32:37] <The_8472> but in the steady state it should minimize wasted connects
[20:37:25] <klapaucjusz> You do enjoy arguing, don't you?
[20:38:03] <The_8472> yes, absolutely
[20:38:10] <klapaucjusz> Replied.
[20:41:25] <The_8472> hrrmmmmm... i think i'm not expressing my concerns properly. but i guess i'll just move along and see where this goes
[20:42:22] <The_8472> soo... do what you think is right and i'll just assist in clarifying things :]
[20:52:34] <The_8472> klapaucjusz, do you have any input on DHT scrapes?
[20:52:44] <The_8472> alus said he'd have to think about it but hasn't commented since then either
[20:53:30] <klapaucjusz> My intuition is still the same.
[20:54:15] <klapaucjusz> I'm alsmost convinced that given a sufficiently smart algorithm, you can do with just having (seeds, leechers) pairs from all tracking nodes.
[20:55:01] <klapaucjusz> I don't think that the amounts of noise you'll see will break your queueing algorithms.
[20:55:27] <klapaucjusz> Util I'm proved wrong, I'm opposed to sending such a complicated data structure (which almost everyone will get wrong) over the wire.
[20:56:52] * alus is moving currently
[20:58:03] <klapaucjusz> alus: you have all my sympathy.
[20:58:15] * klapaucjusz moved in two years ago, and still has one unpacked box.
[20:58:26] <klapaucjusz> s/unpacked/ununpacked/
[21:10:21] *** void^ has quit IRC
[21:10:24] *** void^ has joined #bittorrent
[21:48:10] <The_8472> klapaucjusz, well... a bloom filter isn't all that complicated. just implementing it to yield bit-identical values is important
[21:48:38] <The_8472> for fixed parameters a specification would be quite simple, mostly dictating bit-ordering
[21:48:47] <The_8472> to do that we'd have to agree on the parameters though
[22:04:38] <The_8472> klapaucjusz: http://forum.bittorrent.org/viewtopic.php?pid=908#p908
[22:12:09] <The_8472> also
[22:12:10] <The_8472> http://forum.bittorrent.org/viewtopic.php?pid=909#p909
[22:17:48] *** KyleK_ has joined #bittorrent
[22:22:19] *** HandheldPenguin` is now known as HandheldPenguin
[22:25:03] *** kwinz2 has quit IRC
[22:25:55] *** _rafi_ has quit IRC
[22:30:17] *** Switeck has joined #bittorrent
[22:39:56] *** medecau has quit IRC
[22:43:33] *** kwinz2 has joined #bittorrent
[23:06:52] *** HandheldPenguin is now known as HandheldPenguin`
[23:25:26] *** KyleK_ has quit IRC
[23:29:24] *** kwinz2 has quit IRC
[23:30:16] *** kwinz2 has joined #bittorrent
[23:32:49] *** ukdkbr has joined #bittorrent
[23:39:55] *** kwinz2 has quit IRC
[23:54:09] *** kwinz2 has joined #bittorrent





top