[00:05:39] <The_8472> well, i couldn't debug it. tell me if/when it happens again
[00:05:55] <klapaucjusz> Let's see...
[00:06:37] <klapaucjusz> You did it again as soon as I asked you.
[00:06:42] *** Andrius has quit IRC
[00:06:49] <klapaucjusz> This time, d0d76716175c2598c353c01453844a0f460baabd
[00:07:29] <The_8472> are you sure that it's me? because that's not in my routing table *scratching head*
[00:08:05] <klapaucjusz> Absolutely positive.  You're the only guy on 49001.
[00:08:27] <klapaucjusz> I tried again, and you sent me a different set of dead nodes.
[00:08:57] <klapaucjusz> c04c9679b16568ab26de3c50025312c8216df6c7, d52084687c0095266f454f6d379bccdd48d05dec, 7c5e0868195873b0fbb72f2c7e56440c5f2618c1, 7b53b5dadd79beec944c8f7f03cbf790c3117299
[00:09:10] <The_8472> yup, that is my routing table
[00:09:31] <The_8472> all of them last seen less than 5 minutes ago
[00:10:42] <klapaucjusz> Hmm?  Hold on...
[00:12:18] <The_8472> the D5 one might be dead, but it would take one more refresh here for it to be marked as stale
[00:12:21] <klapaucjusz> Aye, my bug.
[00:12:25] <klapaucjusz> Sorry.
[00:14:57] <klapaucjusz> Fixed, shouldn't happen again.
[00:16:24] <The_8472> ^^
[00:16:50] <The_8472> as i said it would have been possible that you get stale nodes because non-empty buckets can contain non-reachable nodes
[00:17:03] <The_8472> but that all are dead seemed unlikely
[00:17:15] <The_8472> err
[00:17:18] <The_8472> non-full buckets
[00:17:28] *** stalled_ has joined #bittorrent
[00:17:33] <klapaucjusz> Hold on...
[00:17:46] *** hlindhe has quit IRC
[00:17:46] *** hlindhe_ is now known as hlindhe
[00:17:56] <klapaucjusz> I do sometimes have unreachable nodes in my buckets, but I never send them in find_nodes replies.
[00:18:09] <klapaucjusz> You follow?
[00:18:31] <klapaucjusz> Just filter out the non-known-good nodes when formatting the reply.  Piece of cake.
[00:19:11] <The_8472> we don't explicity keep track of reachability. non-reachable nodes just get evicted from full bucket due to bucket maintenance
[00:19:16] <The_8472> it's a rather... indirect process ^^
[00:20:50] <The_8472> like i said, everything is written for the steady state
[00:20:59] <The_8472> with a live DHT all that works just fine -.-
[00:21:07] <klapaucjusz> Mhhmm...
[00:21:20] <klapaucjusz> What data do you keep associated to a table entry?
[00:22:17] <The_8472> ip, port, unix timestamp when it was first seen, unix timestamp when it was last seen, number of failed requests since last successful one
[00:22:39] <klapaucjusz> Okay.
[00:22:43] <klapaucjusz> Then it's easy.
[00:23:00] <klapaucjusz> Add one new field: time when it last gave a reply that made sense.
[00:23:40] *** SundanceKid has quit IRC
[00:23:47] <klapaucjusz> Only send reply if (i) seen more recently then K1, and (ii) gave reply more recently then K2.
[00:23:50] <klapaucjusz> K2 > K1.
[00:24:10] <klapaucjusz> In my implementation, K1=900, K2=7200.
[00:24:41] <klapaucjusz> Other than that, no changes at all to the algorithms.
[00:25:24] * klapaucjusz is happy to say that his Teredo relay is behaving again.  Nothing like a little exercice.
[00:26:17] *** stalled has quit IRC
[00:27:02] <The_8472> hrmhrm, sounds reasonable. i'll see when i get to adjusting the bucket management. it's a bit spaghetti-code-ish right now
[00:27:35] <klapaucjusz> Just add a delegate somewhere ;-)
[00:28:21] <The_8472> no problem that cannot be solved by another layer of indirection. except the problem of too many layers of indirection :)
[00:28:50] *** stalled_ is now known as stalled
[00:28:58] <The_8472> btw, i'm awaiting your response to the DHT scrape thread :)
[00:29:13] <klapaucjusz> Yeah, I need to think it over.
[00:29:21] *** hlindhe_ has joined #bittorrent
[00:29:38] <klapaucjusz> I just don't see how it can work.
[00:29:56] <The_8472> maybe you're thinking of something else than i do
[00:30:19] <klapaucjusz> I doubt it.  I've re-read your message a few times, and I'm pretty sure it's not a misunderstanding.
[00:31:02] <The_8472> my approach is that filled nodes simply don't respond with a token during a get_peers lookup. then the client will not put them in their closest_set and thus backtrack/continue the lookup until it has K closest nodes to the key that returned a token. then it stores on those nodes
[00:31:39] <klapaucjusz> The_8472: I know, I know.  As I've said, I understand what you're saying.  I just believe it's wrong.
[00:31:57] <klapaucjusz> So either I work out why I'm wrong, or I'll write a lengthy message on f.b.o.
[00:32:09] <The_8472> ^^
[00:33:14] <The_8472> in fact, instead of backtracking we can just use the previously found nodes before the K closest ones
[00:33:33] <The_8472> since other peers will stumble over those too as long as they're still reasonably close to K
[00:34:04] <The_8472> not that those sets should be all that different
[00:36:06] <The_8472> well, they start to differ significantly once you get further away from the target
[00:44:08] *** _rafi2_ has quit IRC
[00:45:12] *** _rafi_ has joined #bittorrent
[00:47:49] * klapaucjusz just ended a commit message with \n.
[00:53:02] <yehaT> anyone know the formula which utorrent use to put data into its partfiles
[00:54:53] <yehaT> i think it's something like piece# * piecesize and something else
[01:00:25] <The_8472> oh god... traversing the DHT space sideway requires bignum arithmetic
[01:00:32] <The_8472> or at least something close to it
[01:01:13] <The_8472> good that i've written most of it already for the size estimator ~~
[01:02:27] <klapaucjusz> Huh?
[01:02:55] <klapaucjusz> Why not be more brutal than that?
[01:03:00] <klapaucjusz> How much memory do you have?
[01:03:26] <The_8472> tons
[01:03:57] <klapaucjusz> Then just do a depth-first traversal of the graph, avoiding already-seen nodes.
[01:04:02] <The_8472> backed by several GB of swap if necesasry
[01:04:17] <klapaucjusz> You could probably manage in half a gig.
[01:04:31] <The_8472> i have to calculate which keys to ask for ;)
[01:04:45] <klapaucjusz> No, you don't.
[01:04:45] <The_8472> unless... i just ask for their own node IDs, hah!
[01:05:25] <klapaucjusz> No, be more brutal than that.  Ask each node for two or three random keys, including one that's really close to its own key.
[01:05:46] <klapaucjusz> (A spider should be brutal and fast -- there's only one instance running.)
[01:06:14] <The_8472> asking each node will return its local bucket. then i'll ask each node that i haven't seen yet for their local bucket in turn
[01:06:23] <The_8472> can't be more exhaustive than that
[01:06:34] <klapaucjusz> Yep, you're right.
[01:06:44] <klapaucjusz> And thinking about it -- go for depth first.
[01:06:59] <klapaucjusz> s/depth/breadth/
[01:07:08] <klapaucjusz> Go for breadth-first.
[01:07:28] <The_8472> thx, that'll make things easier :)
[01:10:13] <klapaucjusz> I still think you're serving me some pretty old nodes.
[01:11:34] <klapaucjusz> 7c5e0868195873b0fbb72f2c7e56440c5f2618c1
[01:12:01] <klapaucjusz> d52084687c0095266f454f6d379bccdd48d05dec
[01:13:00] <The_8472> yep, they're pretty old
[01:13:30] <klapaucjusz> As in -- hours ;-)
[01:13:50] <The_8472> one hour at most ^^
[01:13:51] <alus> what are you two up to?
[01:13:51] <klapaucjusz> Okay, I think it's working.
[01:14:07] <The_8472> alus, DHT v6 implementations
[01:14:09] <klapaucjusz> alus: we've just set up a set of boostrap nodes for the IPv6 dHT.
[01:14:17] <klapaucjusz> dht.wifi.pps.jussieu.fr, port 6881.
[01:14:20] <klapaucjusz> Testing right now.
[01:14:34] <alus> oh, on that topic
[01:14:42] <alus> we run a bootstrap for the v4 DHT on router.utorrent.com
[01:14:47] <The_8472> we know
[01:14:54] <The_8472> but that's sortof useless to test v6 stuff
[01:14:57] <klapaucjusz> We know, and we love you for that.
[01:15:02] *** _rafi_ has quit IRC
[01:15:20] <The_8472> in fact, that's what i've been using so far ^^
[01:15:27] <alus> well, it does such a great job of handing out nodes which are alive, that it basically DDoSes any peer which is in the set
[01:15:43] <klapaucjusz> ?
[01:15:52] <The_8472> you should increase K for that particular node
[01:15:57] <The_8472> so that it keeps a larger routing table
[01:15:59] <alus> we're thinking of changing it to just return random nodes
[01:16:02] <The_8472> and hands out nodes at random
[01:16:07] <The_8472> yeah
[01:16:28] <alus> like, remove all the rest of the functionality.
[01:16:34] <klapaucjusz> Excellent idea.
[01:17:18] <klapaucjusz> Add BEP-32 support while you're at it :-)
[01:17:19] <alus> perhaps we could make it answer v6 requests at the same time...
[01:17:29] <klapaucjusz> Great minds meet.
[01:17:36] <alus> we don't have BEP-32 support planned in uT yet
[01:17:39] <klapaucjusz> You're welcome to my code.
[01:17:50] <alus> we need to fix the Teredo MTU problem in uTP first ;)
[01:17:52] <klapaucjusz> darcs get http://www.pps.jussieu.fr/~jch/software/repos/dht
[01:17:58] <alus> oh, thanks
[01:18:08] <alus> is it really non-trivial?
[01:18:15] <alus> given that we have a v4 DHT already
[01:18:28] <klapaucjusz> It will at least be helpful for testing.
[01:18:42] * alus reads BEP-32
[01:18:56] <klapaucjusz> alus: NOT THE ONE ON BITTORRENT.ORG
[01:19:03] <klapaucjusz> It's obsolete.  Arvid is lazy.
[01:19:20] <klapaucjusz> http://www.pps.jussieu.fr/~jch/software/bittorrent/
[01:19:41] <The_8472> you basically have to implement a 2nd routing table that runs mostly independent of the v4 one. that and a little crosstalk stuff
[01:20:13] <alus> klapaucjusz: aw man
[01:20:25] * alus bugs arvid
[01:20:38] <klapaucjusz> Thanks.
[01:26:31] <klapaucjusz> alus: there's no rush with implementing BEP-32.
[01:26:49] <klapaucjusz> On the other hand, I'd be very grateful if you people could review the document, and either agree or disagree.
[01:27:04] *** bittwist has quit IRC
[01:27:07] <klapaucjusz> ...so that we don't have to change the spec in incompatible manners later on.
[01:27:26] <klapaucjusz> I'm hoping to get an implementation into Transmission 1.80.
[01:29:41] <alus> klapaucjusz: hey, nice code
[01:30:18] <klapaucjusz> Thanks.
[01:30:33] <The_8472> damn those safeties -.-
[01:30:44] <The_8472> my keyspace crawler killed itself
[01:30:53] <klapaucjusz> Heh.
[01:30:57] <Switeck> Does Vuze default to using fewer max connections for seeds than regular downloading torrents?
[01:31:11] <The_8472> idk about the defaults, let me check
[01:31:15] <The_8472> personally i do
[01:31:23] <Switeck> It should XD
[01:32:07] <The_8472> hrrm, the option is off by default it seems
[01:32:19] <The_8472> but yes, it should indeed
[01:33:38] <Switeck> If a seed has 100 peers connected and 4 upload slots per torrent...a random peer might download from it ~5% of the time.
[01:34:15] <The_8472> well, that doesn't matter, really
[01:34:46] <Switeck> I cannot imagine that 80+ idle connections for each seed is "useful" or 0 bandwidth cost
[01:34:56] <The_8472> it's just a matter of efficiency, you simply need less connections to keep the upload busy
[01:35:02] <The_8472> yes
[01:35:08] <The_8472> default is 50 though
[01:35:11] <The_8472> not 80
[01:35:14] <Switeck> have messages without Have ALL is nasty
[01:35:49] <The_8472> well, seeds don't care about that, they send the bitfield
[01:36:00] <The_8472> not as efficienct as a have all, but better than tons of haves
[01:36:00] <Switeck> 50 connections per torrent is "standard" for uTorrent "no settings" default settings as well.
[01:36:18] <Switeck> the peers may be sending haves to the seed
[01:36:27] <The_8472> have all won't help in that case
[01:36:35] <The_8472> and in azureus we have stacked haves
[01:36:42] <The_8472> makes things a lot more efficient
[01:37:12] <Switeck> does Azureus back off making outgoing connections if unfirewalled and near connection max?
[01:37:25] <The_8472> yes
[01:37:40] <The_8472> in fact, we disconnect outgoing connections to make room for incoming ones
[01:37:42] <Switeck> (it should continue to max if firewalled though)
[01:37:42] <The_8472> to some extent
[01:37:49] <The_8472> yes, we do
[01:38:02] <Switeck> uTorrent doesn't seem to...
[01:38:19] <Switeck> it will quit making outgoing connections at about 70-80% of max allowed
[01:38:35] <The_8472> even if firewalled?
[01:38:41] <Switeck> yeah :P
[01:38:52] <The_8472> well, suboptimal, but not really harmful
[01:39:05] <Switeck> usually, the limit's set stupid high anyway
[01:39:24] <Switeck> defaults to 50 and only gets increased with about any settings change "recommended" by the setup guide
[01:39:39] * klapaucjusz is filing a bug against transmission, which continues making connections up to its configured limit.
[01:40:31] <Switeck> Transmission used to make 8 incoming connections to me in <20 seconds due to a bug...
[01:41:02] * klapaucjusz is aware that transmission's strategy for opening connections needs a rewrite.
[01:41:25] <Switeck> they fixed that (mostly) after I complained that it should NEVER do that.
[01:41:31] *** init0_ has joined #bittorrent
[01:41:49] <alus> there is no specific logic in uT to stop opening connections at about 70-80% of max allowed
[01:41:49] <The_8472> no, they didn't
[01:41:50] <Switeck> it was v1.33 doing it...(this was some time ago)
[01:41:53] <klapaucjusz> Switeck: transmission does have its problems, no argument about it.
[01:41:59] <The_8472> i complained abotu it recently too, and it's still an outstanding issue
[01:42:16] <The_8472> well, it's better than what you describe. but globally its behavior is still... problematic
[01:42:17] <alus> but it does subtract the number of halfopen from the max allowed, meaning if those all succeed it will be at max
[01:42:18] <klapaucjusz> But it is extremely clean and comprehensible code, and Charles is a guy one can speak with.
[01:42:25] <klapaucjusz> So the issues are slowly getting fixed.
[01:42:33] <Switeck> alus, when I have connections per torrent set to 10, I don't get >7 connected while firewalled
[01:42:48] <alus> Switeck: coincidence
[01:42:58] <Switeck> on 10 torrents/
[01:43:04] <Switeck> over weeks?
[01:43:34] <alus> the extra three connections are either recently dropped due to timeouts, in the halfopen state, or the global limit it hit
[01:43:46] <alus> s/it/is/
[01:43:51] <Switeck> admittedly, the vast majority of ips fail for public torrents
[01:43:59] <Switeck> too many firewalled peers
[01:44:10] <alus> tell the tracker people to do nat check :P
[01:44:34] <Switeck> sure, uTorrent write a decent tracker and people would probably use it XD
[01:44:36] <The_8472> you can't do that at the load levels you see at tpb&co
[01:44:52] <alus> sure you can
[01:45:11] <Switeck> just a firewalled flag sent to tracker...why lie about being firewalled?
[01:45:20] <The_8472> uhm... no. they already have problems with TCP sockets so that they prefer UDP tracker announces
[01:45:49] <The_8472> doing connection checks to possibly firewalled peers (wait for the socket to time out... yay) would simply kill their network stacks
[01:45:58] <The_8472> not to mention that it would more than double their traffic
[01:46:33] <alus> Switeck: http://github.com/bittorrent/Helix
[01:46:57] *** init0 has quit IRC
[01:47:14] <alus> The_8472: so only do as much as you can handle. pinging uT over uTP would be fine, too
[01:47:55] <Switeck> why bother to check? default permit
[01:48:20] <The_8472> anyway, PEX should do the work, trackers are just entry points into the swarm imo
[01:48:24] <Switeck> only false negatives are an issue
[01:48:29] <alus> this tracker http://github.com/bittorrent/Helix by the way is what BT uses internally for Very Large swarms
[01:48:34] <alus> and has NatCheck
[01:49:08] <alus> The_8472: Switeck's report indicates PEX is not doing a great job. we don't specifically report connectability in PEX
[01:49:30] <Switeck> my report?! o.O
[01:49:36] <The_8472> that would be one thing we could improve on without putting a huge burden on trackers
[01:49:43] <alus> <Switeck> admittedly, the vast majority of ips fail for public torrents
[01:49:50] <alus> The_8472: true
[01:49:58] <Switeck> ah
[01:50:11] <The_8472> we don't even have to do a nat check, we just send if it's an incoming our outgoing connection
[01:50:20] <Switeck> that assumes the vast majority of ips are coming in via PEX
[01:50:30] <alus> The_8472: that would be a good first step, but how should one use that information?
[01:50:32] <burris> cool, mit license
[01:50:35] <The_8472> and prefer to connect to peers that are outgoing ones
[01:51:06] <alus> The_8472: so poisoners could obviously use that flag to prioritize their own IPs. maybe not a big deal, though
[01:51:46] <Switeck> it'd make sense...even the poisoners probably have some upper limit on how fast they make outgoing connections
[01:52:21] <The_8472> you probably could do some more elaborate analysis by matching data from different peers or whatevr
[01:52:36] <The_8472> or simply order the pex lists based on incoming/outgoing and connect to the outgoing ones first
[01:52:43] <The_8472> and try incoming later
[01:52:50] <The_8472> so you can ramp up faster
[01:53:04] <Switeck> and ban poisoners quicker XD
[01:53:04] <klapaucjusz> Trouble is, too many NAT-ed hosts.
[01:53:10] <klapaucjusz> If there were just a few, we'd be fine.
[01:53:23] <klapaucjusz> Perhaps we could promote IPv6 ;-)
[01:53:31] <Switeck> When I was doing BearShare beta testing years ago, there was already 70% firewalled
[01:53:34] <The_8472> promote windows 7 then ^^
[01:53:40] <The_8472> ipv6 enabled by default
[01:53:48] <The_8472> and finally a sane dual stack
[01:53:50] <Switeck> I think more people percent-wise use a router or software firewall that auto-blocks now
[01:53:56] <The_8472> instead of the separate stack in XP
[01:53:57] <Switeck> as well as modems now containing routers! ><
[01:53:59] 
[01:54:18] <The_8472> ^^
[01:54:39] <alus> The_8472: Vista had ipv6 enabled by default too
[01:54:40] <The_8472> hey, bittorrent is only doing what... 60% of all internet traffic?
[01:54:46] <The_8472> vista is windows 7 beta
[01:54:49] <alus> klapaucjusz: you're welcome?
[01:55:11] <alus> klapaucjusz: do you feel summoned?
[01:55:21] <alus> The_8472: true
[01:55:22] <The_8472> keyspace crawl: 2253 sent, 812 responses, 1423 timeouts
[01:55:26] <The_8472> this is going to take a while
[01:55:52] <klapaucjusz> alus: where's the invitation to?
[01:56:11] <klapaucjusz> The_8472: that's because of people poisoning the DHT with old nodes ;-)
[01:56:20] <klapaucjusz> You're going to have to increase alpha.
[01:56:39] <The_8472> already doubled it compared to regular lookups ~~
[01:57:05] * klapaucjusz says alpha := 1000
[01:57:09] <alus> The_8472: so bit 1 is encryption, bit 2 is see, bit 4 is uTP, bit 8 is holepunch. shall we use 16 for "outgoing connection"?
[01:57:16] <The_8472> that would fry my router instantanously
[01:57:35] <The_8472> what about the bits inbetween?
[01:57:35] <alus> klapaucjusz: the invitation was by me, for people like you to come join our bandwidth party
[01:57:40] <klapaucjusz> You're doing that behind NAT?
[01:58:02] <The_8472> nvm ^^
[01:58:19] * The_8472 usually thinks in bit positions not their significance
[01:58:51] <klapaucjusz> alus: oh, the bandwidth is not what gets me excited.
[01:59:03] <alus> klapaucjusz: what does?
[01:59:26] <klapaucjusz> It's the look on the face of all the IPv6 gurus who suddenly realise that the killer application they've been speaking about is there, and they're not the ones who made it happen.
[01:59:35] <klapaucjusz> ...and they don't like it a bit ;-)
[02:00:02] <klapaucjusz> (Hold on, I'll try to find the thread)
[02:00:10] <alus> haha. well give them my email
[02:00:31] <klapaucjusz> You're the person guilty for the infamous "enable IPv6" button?
[02:00:35] <alus> yup
[02:00:40] <klapaucjusz> Enabled by default in the betas?
[02:00:49] <klapaucjusz> Alus, I owe you a beer.
[02:01:09] <alus> we tried that, but it wasn't worth it. XP's ipv6 stack is busted
[02:01:25] <alus> and Vista+ has it enabled anyway
[02:01:45] <The_8472> yep, it's the reason why sun doesn't support IPv6 with nonblocking IO on windows... until java 7 -.-
[02:01:54] <The_8472> *rages against XP*
[02:01:59] <Switeck> Teredo not the "Enable IPv6" you're talking about alus?
[02:02:13] <klapaucjusz> alus: "Welcome to your v6-enabled transit network, whether you like it or not"
[02:02:21] <klapaucjusz> http://www.gossamer-threads.com/lists/nanog/users/107692?page=last
[02:02:21] <alus> (specifically, enabling IPv6 would cause DNS resolution to always try AAAA, which often timed out on crappy routers which tried to handle DNS, so using the web suddenly tool 9 more seconds per DNS request than needed)
[02:02:52] <The_8472> can't you fix that by changing the prefix policy?
[02:03:05] <alus> if you can change the prefix policy on XP
[02:03:12] <klapaucjusz> The_8472: no, prefix policy happens after you do DNS resolve.
[02:03:13] <alus> I didn't look in to it
[02:03:25] <The_8472> oh, i thought they use it for DNS too -.-
[02:03:27] <klapaucjusz> Before you DNS resolve, you don't have the prefixes yet.
[02:09:42] <klapaucjusz> Well, now that I've fixed a few remaining bugs in the IPv6 DHT code, it looks like its running fine.
[02:09:50] <klapaucjusz> Although of course we'll need more buckets to be sure.
[02:10:01] <klapaucjusz> Okay, time to put a HOWTO on f.b.o.
[02:13:04] <uriel> K`Tetch: you around?
[02:19:05] *** bittwist has joined #bittorrent
[02:20:14] <alus> The_8472: so, 16s bit? (or bit 5, if you prefer)
[02:20:30] <The_8472> yes, why not.
[02:20:52] <alus> woo
[02:20:55] <The_8472> that should be documented somewhere *cough*
[02:21:00] <alus> well, so should PEX :P
[02:21:15] <The_8472> that's why i'm coughing
[02:22:32] <klapaucjusz> alus: you could at least make a forum.b.o posting with the bit assignments.
[02:22:37] <The_8472> klapaucjusz, btw. how do you come to the 10M nodes estimate for the mainline DHT? the estimator i've written usually gets to values between 4 and 6M
[02:23:13] <klapaucjusz> Wow... I didn't expect to get that close with my back-of-the-envelope estimation.
[02:23:31] <The_8472> of course, i don't quite trust my own code since everything works with powers of two a simple off-by-one error can cause an error by the factor of 2
[02:23:41] <klapaucjusz> Exactly.
[02:23:55] <klapaucjusz> let nb=number of buckets
[02:24:07] <klapaucjusz> (2^nb)*8
[02:24:26] <klapaucjusz> I'm getting 8M, with M=1024*1024
[02:25:01] <Switeck> DHT + PEX might reach even more
[02:25:07] <The_8472> mine works differently. too much noise in the routing table. instead i look at the distance between the nodes of node lookups and convert that to doubles. (everything kept in log2 representation)
[02:25:14] <klapaucjusz> Switeck: that's not the number of peers.
[02:25:20] <klapaucjusz> That's the number of nodes in the DHT.
[02:25:23] <K`Tetch> uriel - yes
[02:25:26] <The_8472> and keep a walking average over the results as i shovel them into the estimator
[02:25:51] <Switeck> how many peers can connect to a node? and roughly what average then?
[02:26:10] <klapaucjusz> BitTorrent and DHT are two distinct protocols.
[02:26:11] <The_8472> every DHT-capable client is a node
[02:26:38] <klapaucjusz> Node: participates in the DHT.  Peer: participates in BitTorrent.
[02:26:49] <Switeck> but every node on DHT represents one BitTorrent client?
[02:26:57] <klapaucjusz> Not necessarily.
[02:27:05] <The_8472> but approximately yes
[02:27:11] <klapaucjusz> The nodes in dht.wifi.pps.jussieu.fr only do DHT, they don't do BitTorrent.
[02:27:27] <klapaucjusz> (Granted, there aren't many of those.  Three right now.)
[02:27:33] <The_8472> they're a vanishing amount
[02:27:41] <The_8472> hence: approximately
[02:27:42] * klapaucjusz expects an apology
[02:27:55] <Switeck> presumably, there are many running BitTorrent clients that are not DHT nodes
[02:28:03] <klapaucjusz> Yep.  Millions.
[02:28:07] <Switeck> possibly even "most"
[02:28:23] <The_8472> bittornado, old mainline
[02:28:29] <klapaucjusz> Not counting all the eegits who disabled the DHT because of the FUD propagated by the private trackers' admins.
[02:28:30] <The_8472> those are the most popular ones without dht
[02:28:38] <The_8472> not sure.. xunlei maybe too
[02:28:46] <Switeck> even the ones with DHT may either have it disabled or fail to connect
[02:29:04] <klapaucjusz> And of course all the Azureus peers that only participated in The Other DHT.
[02:29:16] <The_8472> damnit, i think i have some concurrency problem somewhere
[02:29:19] <The_8472> hey, ours was first
[02:29:37] <klapaucjusz> Time to move on ;-)
[02:29:56] <uriel> K`Tetch: want to post that?
[02:30:01] <Switeck> so long as there's bridges between the 2 DHT's, it matters little now
[02:30:12] <uriel> ( trackon.org )
[02:31:26] <Switeck> I've heard of mainline DHT plugin for Azureus...does that possibly access both DHT networks at once?
[02:31:42] * klapaucjusz can hear The_8472 grinning
[02:36:19] <The_8472> with that plugin azureus can use both DHTs, yes
[02:36:55] <The_8472> all 5 of them actually -.-
[02:37:10] <The_8472> azureus ipv4, ipv6, beta dht. mainline ipv4 and ipv6
[02:37:23] <The_8472> i think my ISP hates me
[02:38:32] <klapaucjusz> That's 4.
[02:39:34] <The_8472> beta is a separate DHT for testing purposes in azureus
[02:39:48] <The_8472> thus 5
[02:46:21] <The_8472> goddamn concurrency issues -.-
[02:47:07] <The_8472> at least we have atomic stuff now, i just have to use it when necessary
[02:47:10] *** waldorf_ has quit IRC
[02:47:24] <klapaucjusz> The_8472: atomic stuff?  Java 7? Cool.
[02:47:31] <The_8472> since java 5 actually
[02:47:36] <klapaucjusz> Pointer?
[02:47:50] <TheSHAD0W> It's atomic?
[02:47:53] <TheSHAD0W> It's atomic!!
[02:48:05] * TheSHAD0W gasps
[02:48:17] <The_8472> http://download.java.net/jdk7/docs/api/index.html?java/util/concurrent/package-summary.html
[02:48:31] <The_8472> and http://download.java.net/jdk7/docs/api/index.html?java/util/concurrent/atomic/package-summary.html
[02:48:51] <TheSHAD0W> (Name that movie.)
[02:49:09] <The_8472> basically convenient wrappers for compare-and-set operations
[02:49:13] <The_8472> and collections based on them
[02:49:26] <The_8472> like concurrent linked lists
[02:49:56] <klapaucjusz> (Java is one of the rare mainstream languages that get concurrency right.  The chapter about concurrency in the JVM spec is a classic.)
[02:50:36] <TheSHAD0W> http://theatomiccheesemelt.blogspot.com/2009/10/movie-review-dr-seuss-5000-fingers-of.html
[02:51:16] <klapaucjusz> AtomicBoolean.lazySet?
[02:52:09] <The_8472> it doesn't provide the ordering guarantees (membars) like set does
[02:52:38] <The_8472> it can be more efficient, but usually is only useful in some finishing operations... idk... to set a variable to false that eventually breaks a loop or something like that
[02:53:38] <klapaucjusz> Yep, makes sense.
[02:55:29] <klapaucjusz> Nice.
[03:03:16] <The_8472> hehe, looks like atomicinteger fixed it
[03:03:27] <The_8472> funny how you only notice such things under high load
[03:16:46] <TheSHAD0W> Email received:
[03:17:15] <TheSHAD0W> Dear sir,
[03:17:15] <TheSHAD0W> I use your program for a couple of years now with a lot of pleasure.
[03:17:15] <TheSHAD0W> I am looking for a bittorrent client for windows media center.
[03:17:15] 
[03:17:15] <TheSHAD0W> Maybe you cam make a windows media version.
[03:17:15] <TheSHAD0W> Thanks
[03:17:44] <TheSHAD0W> Vuze or uTorrent want to jump on it?
[03:18:04] <Switeck> 10-foot pole!
[03:18:11] <TheSHAD0W> LOL
[03:19:40] <The_8472> we already do
[03:19:53] <The_8472> vuze can act as UPnP-AV server for downlaoded content
[03:20:28] <The_8472> including on the fly encoding and streaming
[03:20:30] <The_8472> so... yeah
[03:20:53] <klapaucjusz> The_8472: what's the UDP timeout of your router?
[03:21:15] <The_8472> for conntrack?
[03:21:17] *** waldorf_ has joined #bittorrent
[03:21:45] <The_8472> 120 seconds
[03:22:04] <The_8472> it doesn't have the NOTRACK module
[03:22:06] <The_8472> i wish it had
[03:22:20] <The_8472> then i could make all forwarded ports stateless
[03:23:49] <klapaucjusz> You should do.
[03:24:25] <The_8472> i don't want to setup a crosscompiler just to compile one kernel module... too lazy
[03:25:40] <klapaucjusz> OpenWRT?
[03:25:45] <The_8472> ddwrt
[03:26:51] <klapaucjusz> File a bug.
[03:28:06] <The_8472> with the ddwrt guys? oh my. i had to add a handful of user scripts just to get 6to4 working, because they botched it with one of the releases, knew of it and didn't bother to fix it -.-
[03:28:43] <klapaucjusz> Switch to OpenWRT, then.
[03:28:55] <The_8472> they have notrack?
[03:29:18] <klapaucjusz> No idea.  But they listen to bug reports.
[03:34:16] 
[03:34:40] <alus> klapaucjusz: two things. 1. STUN is not holepunching :P 2. uT already deployed holepunching
[03:35:56] <alus> and I'm certain that more uTorrent peers support uTP holepunching than are running on a version of Windows with Teredo enabled
[03:36:47] <Switeck> so uTorrent v1.8.3 and up support uTP holepunching?
[03:37:56] <klapaucjusz> alus: perhaps.
[03:38:20] <klapaucjusz> This doesn't prevent me from making unjustified claims that you're unable to disprove :-)
[03:38:39] <Switeck> If only uTorrent v1.9 and up support uTP holepunching, I cannot imagine there's more of them than Teredo enabled Windows versions out there...
[03:38:48] <The_8472> good that such things as burden of proof don't exist
[03:38:55] <Switeck> my torrent swarms simply don't reflect that!
[03:39:06] <klapaucjusz> I think the main point is that by working on deploying Teredo, you're helping users of all implementations of BitTorrent.
[03:39:29] <klapaucjusz> By working on a private, unpublished extensions called ut_holepunch, you're only helping BitTorrent users.
[03:39:36] 
[03:39:52] <klapaucjusz> What's more, you're potentially self.shootFoot().
[03:40:45] 
[03:41:17] <klapaucjusz> In short, you're building your own protocol stack at the application layer, rather than adding the features you need in the lower layers, where they belong.  It's shortsighted.
[03:41:43] <klapaucjusz> (Granted, it does give you a competitive advantage.)
[03:41:50] <The_8472> i think i've said so all along. ledbat belongs into tcp. doing a transport protocol at layer 7 is just plain wrong
[03:42:07] <The_8472> klapaucjusz, so does vendor-lockin. doesn't mean it's a good thing
[03:42:17] <klapaucjusz> The_8472: it's STUN we're speaking about, not Ledbat.
[03:42:33] <The_8472> i thought they're not doing STUN
[03:42:38] <klapaucjusz> They are.
[03:42:52] <klapaucjusz> Look at the LTEP extension headers, search for ut_holepunch.
[03:43:02] <The_8472> holepunching is not stun ^^
[03:43:21] <The_8472> stun involves all the cone-ness detection of your router
[03:43:28] <Switeck> alus already gave me the differences about STUN and uTP holepunching XD
[03:43:42] <klapaucjusz> ?
[03:43:53] <klapaucjusz> Ah, okay, I see.
[03:44:03] <klapaucjusz> So they're doing NAT-t.  At the application layer.
[03:44:06] <Switeck> STUN is a method of detecting NAT cones
[03:44:21] <Switeck> NAT-t is just a use of it
[03:46:00] <alus> klapaucjusz: we don't tie holepunching to uTP. we built a BT extension which attempts holepunching, and we only do that with uTP
[03:46:08] <alus> klapaucjusz: so it doesn't affect the uTP protocol at all
[03:47:12] <alus> klapaucjusz: I should also mention we use uTP on platforms which are not Windows. Those platforms need to get their collective Teredo asses in gear
[03:47:21] <klapaucjusz> Agreed.
[03:47:31] <klapaucjusz> May I quote you?
[03:47:49] <alus> haha
[03:47:57] <klapaucjusz> I'm going to publish an extended rant on the state of IPv6 as soon as I get the IPv6 DHT into Transmission.
[03:47:58] 
[03:48:05] <alus> sure
[03:48:13] <alus> The_8472: uh, uT for Mac?
[03:48:24] <alus> http://www.utorrent.com/downloads/mac
[03:48:26] <klapaucjusz> ...and most of the Transmission users are Macies.
[03:48:30] <alus> sure
[03:48:36] <alus> also the BitTorrent SDK
[03:48:45] <alus> which runs on linux powered NAS devices and such
[03:48:49] <The_8472> ah right, there was something. you rarely hear of it ^^
[03:48:58] <klapaucjusz> The_8472: you're underestimating them.
[03:49:04] <alus> it's a rare case, but it was an important design consideration at the time
[03:49:04] <klapaucjusz> They're a pain, but they're really good.
[03:49:53] <klapaucjusz> At the time?
[03:50:41] <klapaucjusz> Fon?
[03:51:24] <alus> one day, after Teredo implementation, we were discussing the BT SDK and the Mac port and such
[03:51:43] <alus> and we decided that punting on holepunching previously and using Teredo was no longer going to work on these new platforms
[03:52:03] <alus> so we came up with (two, and ditched) a holepunching scheme
[03:52:30] 
[03:52:52] <alus> The_8472: then work on TCP implementations.
[03:52:54] <Switeck> you mean like encryption? XD
[03:52:57] <klapaucjusz> The_8472: I think we've said everything we had to say on the subject.
[03:53:06] <The_8472> Switeck, yes
[03:53:23] <klapaucjusz> You're just repeating yourself.  They got it, and they're most likely discussing our points internally.
[03:53:25] <The_8472> klapaucjusz, yes we're probably boring them with it... but still
[03:53:27] <alus> The_8472: I agree with you. We think the best way is to implement in the wild to make sure it works and stopgap as you say, and go to the IETF with ledbat
[03:53:36] <Switeck> I regularly get a BitComet client trying to connect using its proprietary encryption...
[03:53:51] <alus> The_8472: obviously if this was just a plugin anyone could install, I would prefer that to the application-layer implementation
[03:53:55] <The_8472> yeah, all that legacy crud is annoying
[03:54:41] <Switeck> I don't even want to consider getting router and modem makers to implement it in their hardware...
[03:54:44] <The_8472> we even have support for some prehistoric UDP tracker protocl (not the current one) in Az, because apparently someone is still using it
[03:54:54] <alus> Switeck: they would not need to
[03:55:11] <The_8472> router makers should rather implement 6to4
[03:55:20] <The_8472> and IGMPv3/MLD
[03:55:38] <alus> The_8472: even if there is a plugin which works and does not crash, users without admin access will not be able to use it. we'll still probably keep the application layer protocol until MS ships it in Windows or something
[03:55:47] <Switeck> working multicast through the WAN would be nice too XD
[03:55:48] <alus> The_8472: but you don't have to implement it. you can just use the plugin
[03:56:10] <The_8472> Switeck, that's what i just said
[03:56:30] <Switeck> ah, wasn't aware that's IGMPv3
[03:56:35] <The_8472> alus, well... non-admin users shouldn't be all that significant. i mean at home pretty much everyone has admin permissions if necessary
[03:56:51] <klapaucjusz> alus has a point -- you don't make things happen without first having a working implementation.
[03:57:01] <The_8472> igmpv3 and mld are necessary for source specific multicast. the only sane way to do multicast for p2p
[03:57:08] <The_8472> klapaucjusz. vegas?
[03:57:30] <Switeck> I think encouraging more average users to be running in M$'s insecure admin level...maybe isn't the right way to go forward.
[03:57:39] <klapaucjusz> So why don't you people implement p2p over multicast, and then we'll work on multicast getting deployed.
[03:57:41] <The_8472> you can't say that vegas was deployed to millions of nodes before it was implemented into all major OS stacks
[03:58:09] <klapaucjusz> "all major OS stacks"?
[03:58:24] <klapaucjusz> Vegas is only in Linux, AFAIK, and it's disabled by default.
[03:58:40] <The_8472> windows has compound tcp, which is akin to vegas afaik
[03:58:47] <alus> The_8472: but this isn't applicable to all TCP connections, only those who enabled it. an OS vendor would have to be suicidal to make it default for all TCP connections
[03:58:51] <klapaucjusz> BTW, Vegas doesn't work.  Microsoft uses CTCP, which improves on Vegas quite a bit.
[03:59:22] <alus> The_8472: so the motivation for putting it everywhere all at once is not as great
[03:59:38] <klapaucjusz> alus: http://trac.transmissionbt.com/attachment/ticket/2338/congestion-control.patch
[04:00:12] <klapaucjusz> You've seen that already, I think.  Why don't you go bother Microsoft to do something similar?
[04:00:33] <klapaucjusz> You're single-handedly responsible for 30% of Internet traffic.  They will listen.
[04:00:47] <alus> klapaucjusz: sure. they would also need to implement ledbat
[04:00:49] <alus> or someone would
[04:00:59] <klapaucjusz> Try to get in touch with Christian Huitema, he's a smart guy.
[04:01:21] <alus> MS is showing up to the IETF meetings, I assume that's the proper channel for this sort of change in Windows
[04:01:28] <klapaucjusz> Who?
[04:01:43] <alus> you'd have to ask Stanislav
[04:03:48] *** The_8472 has quit IRC
[04:04:08] *** wadim has joined #bittorrent
[04:04:10] *** wadim is now known as The_8472
[04:04:47] <The_8472> hrmrgl
[04:05:01] * The_8472 hates the 24h forced disconnect
[04:07:01] <The_8472> btw, my DHT crawling is humming along at 22k sent packets and counting
[04:07:11] <The_8472> with an alpha of 100
[04:12:03] <klapaucjusz> Those platforms need to get their collective Teredo asses in gear
[04:12:10] <klapaucjusz> I just love this quote.
[04:12:55] * The_8472 loves java's ability to replace code at runtime
[04:15:40] <klapaucjusz> The_8472: you just served me with 7b53b5dadd79beec944c8f7f03cbf790c3117299
[04:15:46] <klapaucjusz> That's *wrong*.
[04:17:17] <The_8472> how so?
[04:18:26] <klapaucjusz> It's been dead for two hours.
[04:18:32] <klapaucjusz> Or more.
[04:18:50] <The_8472> last responded: 96sec
[04:19:14] <klapaucjusz> Eh?
[04:19:19] <The_8472> last responded: 13s
[04:19:25] <The_8472> it's alive
[04:19:28] <klapaucjusz> No, no.
[04:19:44] <klapaucjusz> That's 5a5669673456cbdb9bf3fd074fcf15da1e9bb5b9
[04:19:50] <klapaucjusz> It took over the IP address.
[04:19:58] <klapaucjusz> You're not parsing node_id?
[04:20:07] <The_8472> hrrm, i actually have logic detecting node id and ip changes
[04:24:10] <klapaucjusz> Don't worry, I've also found a bug in my code... I'm losing one node somewhere.
[04:24:57] <The_8472> the IP-change logic is extremely defensive because i've seen incoming requests with identical node IDs from IPs on different continents (according to geoip) than the original node
[04:25:10] <The_8472> node change logic should be more lenient
[04:25:16] <The_8472> *ID change
[04:25:36] <klapaucjusz> Heh... the solution is to not perform id change checks.
[04:25:50] <klapaucjusz> Index your tables by id.
[04:26:09] <klapaucjusz> Allow two ids to be associated to the same IP.
[04:26:20] <klapaucjusz> When a node changes id, the old id will expire naturally.
[04:26:43] <The_8472> we index by ID, but we also comb through all buckets for identical addresses even when the ID differs. theoretically
[04:27:30] <The_8472> ahh, it's even more tricky
[04:27:40] <The_8472> return nodeID.equals(other.nodeID)
[04:27:41] <The_8472> 					|| addr.getAddress().equals(other.addr.getAddress());
[04:29:13] <The_8472> bahhh
[04:29:15] <The_8472> as usual
[04:29:22] <klapaucjusz> Found it!
[04:29:24] <The_8472> node ID detection only works on full buckets -.-
[04:29:46] 
[04:29:48] <The_8472> *haul
[04:30:59] <The_8472> more accurately
[04:31:06] <The_8472> it only works when we actually have to ping the node
[04:31:26] <The_8472> due the small routing table its so busy that we never have to ping it
[04:32:26] <klapaucjusz> * Fix a bug that could cause us to send incomplete lists in response to find_nodes.
[04:33:57] * klapaucjusz suspects somebody is struggling with spaghetti code.
[04:34:10] <klapaucjusz> ;-)
[04:34:12] *** evil_andy has joined #bittorrent
[04:34:17] <The_8472> yes, it's ugly
[04:34:43] <The_8472> the funny thing is, everything works as it should on the v4 DHT
[04:34:52] <The_8472> damn those edge cases
[04:35:03] <klapaucjusz> Yeah, same thing here.
[04:35:13] *** evil_andy has left #bittorrent
[04:35:16] <klapaucjusz> I'd never have noticed the missing node in reply to find_nodes over IPv4.
[04:35:46] <The_8472> you're short by one?
[04:37:29] <klapaucjusz> No, fixed now.
[04:37:39] <klapaucjusz> Removing duplicates was buggy.
[04:38:13] <klapaucjusz> Please send me find_nodes(want=n6)
[04:39:04] <klapaucjusz> Thanks.
[04:43:38] <klapaucjusz> Yep, looks like it's fine.  Mmhh, DHT.
[04:48:03] <The_8472> this is slooow....
[04:48:07] <The_8472> 50k and counting
[04:48:19] <klapaucjusz> Another 8 million to go.
[04:48:28] <The_8472> T.T
[04:48:52] <klapaucjusz> Strange.
[04:49:01] <klapaucjusz> Now you're reachable from Teredo, but unreachable from native.
[04:49:23] <The_8472> my ip changed in case you didn't notice
[04:49:49] <klapaucjusz> Ah, that's it.  No, I hadn't noticed.  Fine.
[04:50:08] <The_8472> it's 2002:5936:b787:1:29be:f1fa:5975:fe66 and 2001:0:4137:9e50:2c47:ea2:a6c9:4878 now
[04:51:07] * The_8472 is going to miss the good old ipv4 days where we could easily memorize a dozen addresses or so
[04:51:36] *** waldorf_ has quit IRC
[04:52:05] <klapaucjusz> I've been working with IPv6 since before Teredo...
[04:52:17] <DeHackEd> I think dynamic DNS services will become even more popular in the future...
[04:52:23] <klapaucjusz> ...and there's a trick: you should memorize just the last digits.
[04:52:40] <klapaucjusz> I.e. when you speak with someone, you say "who's fe66?"
[04:53:04] <The_8472> that only works because i turned address randomization off
[04:53:07] <klapaucjusz> DeHackEd: zeroconf/bonjour.
[04:53:14] <DeHackEd> across the internet?
[04:53:21] <DeHackEd> and avahi-daemon is banned on all my systems.
[04:53:32] <klapaucjusz> Across the Internet you use DNS, right.
[04:53:43] <klapaucjusz> But now imagine telneting into the new printer you just bought...
[04:54:17] <DeHackEd> DHCP supports DNS updates
[04:54:38] <DeHackEd> I'm the expception to the rule and run my own private DNS server, but yes local discovery will become popular...
[04:54:42] <klapaucjusz> Yeah, right.  Tell me when you find a printer with DHCPv6 support.
[04:54:51] <DeHackEd> you have a printer with ipv6 support?
[04:55:13] <klapaucjusz> Interesting question... let me see.
[04:55:32] <The_8472> the printer will probably just use stateless autoconfiguration
[04:55:38] <klapaucjusz> + eth0 IPv6 HP Color LaserJet CM2320fxi MFP (2B4033)      PDL Printer          local
[04:56:00] <DeHackEd> DHCP will still remain popular for auditing...
[04:56:55] <klapaucjusz> It's the only one -- the remaining printers have only IPv4.
[04:57:25] <The_8472> thinking of it... stateless autoconfiguration doesn't push DNS, does it?
[04:57:39] <DeHackEd> not to my knowledge..
[04:58:01] <The_8472> well, there's your purpose for DNS
[04:58:06] <The_8472> errr, DHCP
[04:58:20] <DeHackEd> can't believe I forgot about that...
[04:58:35] <klapaucjusz> Well, there's nothing preventing you from announcing your stateless address to DNS.
[04:58:36] <DeHackEd> and here I was just arguing about using it for DNS configuration (as in, add to a dynamic zone)
[04:58:44] <klapaucjusz> It's just not done in current implementations.
[04:59:16] * klapaucjusz hasn't found a bug in 30 minutes.
[04:59:47] <The_8472> the opposite could probably be said about some other implementations
[05:00:03] <The_8472> "i haven't found a single correctly working feature in 30 minutes"
[05:00:28] <klapaucjusz> Okay, I think I can ship :-)
[05:00:34] <The_8472> yay
[05:01:30] <klapaucjusz> cd transmission; cp ../dht/dht/*.[ch] third-party/dht/; make -j3
[05:03:47] <klapaucjusz> Yay!  A five node DHT!
[05:06:20] <klapaucjusz> Oh, good.  Found a bug.
[05:06:54] <klapaucjusz> Would only trigger on overflow... we're not quite there yet ;-)
[05:16:50] *** klapaucjusz has quit IRC
[05:43:06] *** kmels has joined #bittorrent
[05:48:14] <kmels> hi, how do i get bittorrent to use python 2.4 and not an earlier version?
[05:48:26] <kmels> installed it from a rpm binary
[06:25:53] *** goussx has quit IRC
[06:25:58] *** goussx_ has joined #bittorrent
[06:26:08] *** goussx_ is now known as goussx
[06:28:30] *** _rafi_ has joined #bittorrent
[07:02:18] *** MassaRoddel has quit IRC
[07:05:20] *** RobertX has joined #bittorrent
[07:15:57] *** RobertX has left #bittorrent
[07:17:26] *** MassaRoddel has joined #bittorrent
[07:20:49] *** kmels is now known as kmels-away
[07:27:02] *** Snoopotic has quit IRC
[07:55:06] *** Andrius has joined #bittorrent
[08:03:17] *** kmels-away has quit IRC
[08:32:50] *** stalled_ has joined #bittorrent
[08:32:51] *** cyb2063 has joined #bittorrent
[08:48:17] *** stalled has quit IRC
[08:49:23] *** stalled has joined #bittorrent
[08:53:05] *** stalled_ has quit IRC
[08:59:15] *** stalled_ has joined #bittorrent
[09:07:06] *** stalled has quit IRC
[09:07:07] *** stalled_ is now known as stalled
[09:09:00] *** bittwist has quit IRC
[09:33:53] *** Switeck has quit IRC
[09:45:51] *** burris has quit IRC
[09:48:42] *** bittwist has joined #bittorrent
[10:12:17] *** goussx_ has joined #bittorrent
[10:13:48] *** Miller` has quit IRC
[10:14:09] *** goussx has quit IRC
[10:14:09] *** goussx_ is now known as goussx
[10:42:02] *** Miller` has joined #bittorrent
[10:48:26] *** Miller` has quit IRC
[11:23:17] *** yehaT has quit IRC
[12:10:10] *** kmels-away has joined #bittorrent
[12:34:08] *** betadocto has joined #bittorrent
[12:34:30] *** betadocto is now known as betadoctor
[13:31:50] *** kmels-aw1y has joined #bittorrent
[13:33:49] *** mxs has quit IRC
[13:34:18] *** mxs has joined #bittorrent
[13:43:56] *** kmels-away has quit IRC
[13:45:43] *** The_8472 has quit IRC
[13:46:03] *** wadim has joined #bittorrent
[13:46:05] *** wadim is now known as The_8472
[14:27:10] *** kmels-aw1y has quit IRC
[15:12:55] *** The_8472 has quit IRC
[15:13:16] *** wadim has joined #bittorrent
[15:13:18] *** wadim is now known as The_8472
[15:14:21] <BentMyWookie> does anyone know what client -SP36037 is?
[15:18:29] <The_8472> bitspirit
[15:19:43] <BentMyWookie> cool
[15:20:58] <BentMyWookie> wasn't it -BS ?
[15:21:13] <The_8472> that was prior to version 3.6
[15:21:18] <BentMyWookie> gah
[15:51:17] *** burris has joined #bittorrent
[16:34:54] *** betadoctor has quit IRC
[16:45:28] <The_8472> hah... i actually have a case where goto (well... almost goto) makes code actually more readable
[16:46:32] <alus> what!
[16:46:37] <alus> I don't believe you
[16:46:51] <alus> although klapaucjusz might. I saw him use goto
[16:47:40] <The_8472> java doesn't allow goto. the closest thing you can get is a break <label>; on a code block (most code blocks can have labels) that returns from that code block
[16:48:00] <The_8472> so i can only goto in the forward direction and only do so by leaving a block
[16:48:19] <The_8472> but it's useful to do a series of tests. if one test fails... do the next
[16:48:24] <The_8472> until one succeeds
[16:49:13] <alus> don't we call that "return"?
[16:49:22] <alus> and don't we call those "labels", "function names"?
[16:49:25] <The_8472> return returns from the method
[16:49:41] <The_8472> and those tests are 2-3 lines of code
[16:49:52] <The_8472> operating on the same variables
[16:49:53] <The_8472> over and over
[16:50:02] <The_8472> declaring half a dozen methods would be overly verbose
[16:50:06] <alus> and Java doesn't have closures yet?
[16:50:13] <The_8472> they're working on it
[16:51:05] <The_8472> it's only the 2nd time i actually had to use that feature. both in similar contexts: complex maintenance logic for a queue/cache
[16:52:39] <The_8472> like... check if the entry exists. if found do some consistence checks, if no error, move it to the end. if it doesn't exist, look for a stale entry.
[16:52:42] <The_8472> and the list goes on
[16:53:16] <The_8472> really, it's more readable than before ^^
[16:53:50] <mpl> If you want to go somewhere, goto is the best way to get there. "Ken Thompson"
[16:53:53] <mpl> :D
[16:54:10] <alus> if (found) { if (consistence_checks()) { move_it(); } } else { look_for_stale(); } ?
[16:54:41] <The_8472> alus, sure. if i create an inner class to carry both values (old and new)
[16:55:11] <alus> if you what a who to when now?
[16:55:22] <The_8472> lol
[16:56:04] <The_8472> http://pastebin.com/m4a49b973 <- case 1
[16:56:58] <The_8472> http://pastebin.com/mc5acb39 <- case 2
[16:57:55] <alus> ick
[16:59:05] <The_8472> sure, there are other ways to write it, but not without increasing the verbosity of the code
[17:02:51] <alus> mm. I think I disagree
[17:04:21] <alus> on case 1
[17:04:36] <alus> there are only a two ways toPing can be set
[17:04:49] <alus> but you have a lot of breaks out of the block
[17:04:59] <alus> instead of just returning
[17:05:34] <The_8472> hrrm, true. there used to be more of them
[17:06:06] <alus> so you could remove the block, return in all those places, and pingQuestionable after the if (youngest.getCreationTime() > toInsert.getCreationTime()) { toPing case 1 } else { toPing case 2 } pingQuestionable(toPing);
[17:07:53] <The_8472> hrmhrm
[17:09:00] <The_8472> i'd just return the toping then to a wrapper method
[17:09:58] <alus> for paste case 2, you have an inner break aging; inside break weightedRemove; which is just checked again anyway
[17:10:16] <alus> it looks like you check the same condition a lot, when
[17:10:20] <alus> maybe it hasn't changed
[17:10:37] <The_8472> any removal may change it
[17:11:17] <The_8472> and the breaks bubble up, they break the inner loops too, that's the point
[17:11:44] <alus> sometimes you break all the way out though, when there's a check coming up after the look anyway
[17:12:13] <alus> s/look/loop/
[17:12:16] <alus> time for sleep
[17:12:18] <The_8472> different checks
[17:12:54] <The_8472> gn then
[17:13:38] <alus> are they? 30 and 44, 53 and 64
[17:14:47] <The_8472> well, in that case i would break the loop on that condition and then jump out of the block on the same condition
[17:14:52] <The_8472> so i just jump out directly
[17:16:52] <The_8472> i could shuffle them around a bit, to the end of the loop to merge some checks
[17:17:03] <The_8472> but that wouldn't remove the necessity for all the labels
[17:53:04] *** chelz has joined #bittorrent
[17:57:31] *** waldorf_ has joined #bittorrent
[18:07:44] *** KyleK_ has joined #bittorrent
[18:29:03] *** goussx has quit IRC
[18:49:50] *** Switeck has joined #bittorrent
[18:56:30] <MassaRoddel> which clients cut the dht transaction ids down to 3/2/1 bytes?
[18:58:25] <The_8472> what do you mean by "cut down"?
[18:59:48] *** goussx has joined #bittorrent
[19:00:07] <MassaRoddel> i send 4 bytes and the response is only 3-1 bytes
[19:01:42] <The_8472> hrrm, idk then.
[19:02:01] <The_8472> our plugin had a similar bug, but it simply didn't accept anything but 2 bytes. but that was ages ago
[19:05:26] *** bt42 has joined #bittorrent
[19:20:04] *** waldorf_ has quit IRC
[19:28:01] *** bittwist has quit IRC
[19:35:53] *** waterfoul has joined #bittorrent
[19:36:33] <waterfoul> Which client would you recommend for linux without using wine?
[19:37:12] <waterfoul> Ideally it would have sorting by tracker
[19:38:21] <chelz> not sure about supporting sorting by tracker, but i'd check out transmission, deluge, azureus
[19:39:31] *** kmels-away has joined #bittorrent
[19:39:40] <waterfoul> I didn't like azureus very much (java made it really clunky) I am currently using deluge and am having issues with it not hashing
[19:40:08] <Switeck> you trying to hash files on a NAS?
[19:40:38] <chelz> there's also ktorrent
[19:41:05] <waterfoul> no its a raid array, I have almost 300 torrents and it wont hash the last 10 or so at startup for some reason
[19:41:16] *** A9 has joined #bittorrent
[19:41:21] *** A9 has quit IRC
[19:41:57] <chelz> you might look into rtorrent, it's a bit less fancy than some other clients but it's fast and very efficient
[19:43:31] <chelz> i think you would like ktorrent
[19:45:10] <waterfoul> in ktorrent is there a way to view by tracker?
[19:45:32] <chelz> not sure. but it seems to have a lot of general features.
[19:45:42] <waterfoul> i'll look at it
[19:45:53] <chelz> yeah if that doesn't work then either what you want really sounds to be like either vuze or utorrent
[19:46:04] <The_8472> <waterfoul> I didn't like azureus very much (java made it really clunky) I am currently using deluge and am having issues with it not hashing <- hrm? what's the problem?
[19:46:24] <The_8472> also, java is fast these days. unless you use some crud like gcj
[19:47:02] <chelz> i've been meaning to switch from suns' jre6 to openjdk just to see if stuff i use works
[19:47:28] <waterfoul> often times it gets stuck hasing between files and just stops hashing (I have almost 300 torrents) and I will get a "Too many open files" error sometimes
[19:47:32] <chelz> i do still notice a bit of waiting for the jre to start up, but after it seems to work fine. except a lot of interfaces like to lag for some reason. it's been a while since i've tried az
[19:47:38] <The_8472> hrrm. openjdk still has some issues though
[19:47:50] <chelz> waterfoul: for efficiency rtorrent can't be beat
[19:48:08] <chelz> also as part of the xbt tracker project they have a torrent client in c.. but i think it has near zero features
[19:48:26] <The_8472> <waterfoul> often times it gets stuck hasing between files and just stops hashing (I have almost 300 torrents) and I will get a "Too many open files" error sometimes <- that's a linux issue. you just have to increase the max. file handles for your user
[19:48:37] <The_8472> something with uname -i ... or whatever
[19:49:32] <chelz> ulimit
[19:49:37] <The_8472> ah, right
[19:49:38] <chelz> also found this http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/
[19:49:43] <The_8472> there are other ways to change it
[19:50:00] <chelz> that recommends 'sysctl' but it might be centos/rhel only
[19:50:21] <The_8472> also, if you have 300 torrents then tweaking the JVM parameters might make things more snappy
[19:50:46] <waterfoul> well I used azureus in the past and am currently running deluge
[19:51:11] <The_8472> well, can't help you with that
[19:51:32] <DWKnight> what sort of connection are you actually using?
[19:52:22] <chelz> by my calculations there should be at minimum 5 KiB/s per peer of your upload
[19:53:39] <chelz> so assuming one leecher connected per torrent and 80% upload cap, 1875 KiB/s upload
[19:53:47] <chelz> (300 * 5) / .8
[19:54:06] <The_8472> he didn't say that he had 300 active torrents, did he?
[19:54:15] <chelz> he didn't
[19:54:30] <chelz> but back when i had that many torrents i'd have them all going
[19:54:45] <The_8472> that's quite inefficient
[19:55:09] <chelz> well i'd have a sane max active torrent amount
[19:55:15] <chelz> so most of them would be queued
[19:55:19] <waterfoul> well when most of them are just sitting there waiting for a peer.....
[19:55:30] <The_8472> then you didn't have them all going ^^
[19:55:33] <chelz> yeah that's usually the usage case for having that stuff active
[19:55:38] <chelz> The_8472: hah alright, true
[19:55:38] <waterfoul> I usually don't have more than 10 active at a time
[19:55:51] <waterfoul> I set them all to seeding but they just sit with the seeding status
[19:56:07] <chelz> yeah for that usage, where leechers are far apart timewise it makes sense
[19:56:13] <chelz> poor trackers getting hammered though
[19:56:13] <The_8472> why not just sort by peer count or seed:peer ratio and have most of them queued?
[19:56:30] <The_8472> that's what scrapes + seeding queues are good for
[19:56:48] <waterfoul> I actually didn't understand how deluge did that, could you explain?
[19:56:51] <Switeck> private tracker?
[19:56:56] <waterfoul> yes
[19:56:57] <chelz> The_8472: if you're trying to upload a lot (gain ratio basically) you don't want the lag between your client deciding a torrent should become unqueued and it starting compared to just being ready to go at first
[19:57:01] <The_8472> private trackers have scrapes too
[19:57:03] <chelz> yeah, i guessed as much
[19:57:11] <chelz> eh
[19:57:20] <waterfoul> see what chelz is saying is why I set them to seeding
[19:57:36] <waterfoul> The comp isn't doing much usually so I didn't see the downside
[19:57:40] <chelz> always seemed like black magic how at least az decides to unqueue a torrent
[19:57:56] <The_8472> well, it's all under the seeding rules
[19:58:06] <The_8472> if you sort by seeding rank it's quite obvious actually
[19:58:10] <chelz> waterfoul: well in terms of seeding and using a large amount of torrents, it would be in your best interest to learn rtorrent. especially with torrent grouping so you can set per-tracker requirements.
[19:58:50] <waterfoul> ok, I will look into it
[19:59:00] <The_8472> anyway, i consider the system of running all torrents just to get peers horribly broken. that's what a queue+scrapes are good for. if it's not fast enough for you ask the tracker admin to set the scrape interval to 5 minutes or so
[19:59:01] <chelz> if ktorrent doesn't work that is
[19:59:19] <The_8472> a peer will surely stay on a torrent more than 5 minutes. if it doesn't then you won't get to upload much anyway
[19:59:41] *** btipling has joined #bittorrent
[19:59:43] <chelz> The_8472: what is really wasted or lost by having lots of torrents running rather than queued?
[19:59:53] <waterfoul> With either deluge, rtorrent, or ktorrent is there a way to get the client to take advantage of a multi core processor while hashing
[20:00:21] <DWKnight> are you actually CPU bound with your hashing or are you hard drive transfer bound?
[20:00:30] <DWKnight> most modern systems are hard drive transfer bound
[20:00:40] <waterfoul> cpu bound, the torrents are stored on a raid 5 array
[20:00:48] <chelz> i see no mention of any of those clients having any kind of multicore support
[20:00:50] <Andrius> The_8472, the peer can download torrent faster than in 5 minutes where I live, with private tracers speed is sometimes limited by hdd and not isp...
[20:00:52] <The_8472> well, Az has multithreaded hashing
[20:01:15] <The_8472> well, the private tracker concept fails then if you only have a 5 minute window to upload to someone
[20:01:17] <chelz> az might be the way. although rtorrent hardly hashes stuff
[20:01:19] <The_8472> it punishes non-presence
[20:01:31] <The_8472> instead of punishing leeching
[20:02:06] <chelz> The_8472: it's broken in more ways than that.. but needing ratio is a fact of life and the only way to sustain access to really new torrents being unbelievably fast
[20:02:16] <waterfoul> I have actually seen rewards for setting torrents to seeding on some trackers so it seems that pretty much universally having presence is preferred
[20:02:30] <The_8472> chelz, having a fat upload pipe works perfectly well on public trackers
[20:02:41] <The_8472> i've done 100Mbit download speed on public torrents with ease at my university
[20:02:55] <waterfoul> the thing about private over public is the number of peers
[20:02:58] <DWKnight> having a thin upload pipe managed sanely does as well as having a fat upload pipe
[20:03:32] <waterfoul> on a private tracker it is had to find something with a high peer to seed ratio so time is usually of the essence for snatching the upload from others
[20:03:34] <Andrius> I also often max out my download/upload on public trackers
[20:03:47] <waterfoul> on a public tracker it is easy
[20:03:55] <chelz> The_8472: i'm not doubting that good speeds can be had on public torrents, but it's much more reliable to always get very fast speeds from a popular private tracker
[20:04:16] <The_8472> waterfoul, i would consider that a flaw. because it's unfair to people who are online during different times of day and stuff like that. it penalizes things that have nothing to do with actually seeding or not
[20:04:47] <waterfoul> I agree but it is hopw the system is set up and it seems to work
[20:04:54] <waterfoul> it is flawed but not broken
[20:04:56] <chelz> i definitely max out my upload of 200 KiB / s on public trackers but it's usually hit or miss with my download of 1,536 KiB /s
[20:05:19] <The_8472> it's broken for people who have a slow upload speed and thus can't seed much in the N-minute window
[20:05:32] <chelz> quite broken ;/
[20:05:49] <The_8472> public trackers allow you to leech if there's ample of spare bandwidth. and you download slower anyway if there aren't enough seeds and you upload slow
[20:06:14] <The_8472> there is ONE thing that is actually a good thing about private trackers. they encourage seeding after downloading
[20:06:22] <Andrius> on private trackers bittorrent tends to work differently, as in there's little "tit-for-tat" on torrents that are more than a day old
[20:06:30] <chelz> The_8472: that's just it
[20:06:42] <chelz> Andrius: very true
[20:07:03] <chelz> the goal seems to be to keep retention high, which requires seeders to stick around for a long time on a torrent. the popular enforcement methods of just counting upload vs download really doesn't do a good job towards this goal.
[20:07:07] <The_8472> chelz, ratio-accounting is not the solution then. just keeping track of who quits a torrent immediately after downloading would be a much better approach
[20:07:10] <Switeck> Private trackers have no tit-for-tat essentially
[20:07:19] <The_8472> charles, yes
[20:07:35] <The_8472> ehh... chelz i mean
[20:07:55] <waterfoul> that is why a lot of private trackers implement a bonus system but the bonus systems are hard to keep cheaters out of
[20:08:20] <chelz> yeah hit and run, but if you're going for long retention time then there should be some reward for sticking around for days and days
[20:08:22] <Switeck> tit-for-tat is almost a rare condition XD
[20:08:25] <Andrius> Switeck, on new torrents they do, but it disappears when few fast uploaders finish the torrent
[20:08:46] <The_8472> chelz, sure. but keepign torrents queued and joining when necessary would do the job
[20:08:57] <chelz> when you have 20-50 people seeding on 1Gbit lines it's really hard to get ratio from a home connection
[20:09:00] <The_8472> you don't have to actively seed them if there is no peer
[20:09:01] <burris> the term for two peers that get into a long term reciprocal uploading relationship is "sixty nine"
[20:09:04] <Switeck> 2 sequential downloaders never tit-for-tat with each other.
[20:09:23] <chelz> The_8472: yeah but so far i haven't heard of any incentive systems targeting that. not just good will and general altruism.
[20:09:47] <chelz> ideally there would be some public protocol that enforced or rewarded people for seeding after finishing
[20:09:48] <burris> peers use tit-for-tat to decide who to upload to, but they get into 69 with other peers
[20:10:20] <chelz> i haven't heard that, good term
[20:10:21] <burris> though that term never made it into the protocol doc
[20:10:28] <Switeck> burris, you mean 69 isn't constant uploading...but rather reterning the earlier favor/s?
[20:10:29] <chelz> haha
[20:10:35] <The_8472> it's the first time i hear it too
[20:10:45] <burris> Switeck, no it's simultaneous uploading to each other
[20:11:06] <Switeck> I'd consider that pretty rare on many torrents.
[20:11:24] <The_8472> uhh, that's quite normal during some phases of a publci torrent
[20:11:35] <chelz> carrying stats over from one torrent to another in some kind of public system. i think it would have to be decentralized in some way since the load on a tracker would be very high and a larger single point of failure.
[20:11:51] <Switeck> firewalled peers, peers with the same percentage, peer/seed, etc...
[20:11:56] <burris> yeah, it's pretty common in the initial seeding stage if the seeder isn't super slow
[20:12:14] <The_8472> <burris> Switeck, no it's simultaneous uploading to each other <- last time i checked we simply called that "a reciprocating peer"
[20:12:17] <Switeck> I've seen feast-or-famine in single seed stages of a torrent
[20:12:33] <The_8472> Switeck, firewalling doesn't keep you from uploading. TCP is bidirectional
[20:12:34] <burris> The_8472, yeah but bram calls it 69
[20:12:42] <Switeck> the seed is too slow for the swarm, so you get a short burst that fills the single new piece in
[20:12:53] <The_8472> burris, i haven't heard anything from bram in ages....
[20:12:54] <Switeck> rest of the time you sit on 0 DL, 0 UL
[20:13:14] <Andrius> I sometimes wish bt clients would try to "return earlier favors" more, it's annoying to see a peer that you have fat pipe with, but bt client just tries other peers instead...
[20:13:16] <chelz> last i heard he was working on a streaming p2p thing
[20:13:31] <Switeck> The_8472, at this present time 2 firewalled peers "seldom" upload to each other.
[20:13:46] <chelz> Andrius: bittyrant might do more of what you're thinking of
[20:13:50] <The_8472> Andrius, no. that is bad behavior. we try to only do that to a very small degree because network conditions can change in unpredictable ways
[20:14:11] <The_8472> BT is rate tit-for-tat instead of block tit-for-tat for a reason. blockwise it simply fails
[20:14:30] <Switeck> and I probably should clarify that same availability percentage (same parts)
[20:14:47] <waterfoul> well thanks for the help guys
[20:14:55] <The_8472> Switeck, ah well. yes. that is a scenario if you have a slow uploader
[20:15:04] *** waterfoul has left #bittorrent
[20:15:09] <Switeck> it can also be the scenario with a fast uploader
[20:15:11] <Andrius> yeah, but sometimes network conditions don't change, peers simply exchange all the data they have with each other and have nothing to upload to each other. Then when they have something for each other again, they don't get unchoked
[20:15:21] <Switeck> because they have infinite upload slots :P
[20:15:33] <The_8472> that doesn't keep peers from doing random requests
[20:15:39] <The_8472> and thus everyone downloads a different subset
[20:15:42] <The_8472> and can still share with other peers
[20:16:19] <Switeck> sure but when the lone seed is trying to upload to 100+ peers at once, even if the seed is pretty fast the time between new pieces appearing is non-trivial.
[20:16:50] <The_8472> Andrius, in azureus we have a slight bias in the optimistic unchokes for that case. but if you do it too strong then you get locked into the nodes that used to be fast, even if there might be newer, better peers now
[20:17:14] <The_8472> Switeck, sure. but then several new pieces might appear at once
[20:17:26] *** ajaya has joined #bittorrent
[20:17:27] *** leotr has joined #bittorrent
[20:17:38] <Switeck> yes, and as a matter of fact they usually arrive in clusters
[20:17:51] <Switeck> but the swarm quickly digests them and goes right back to nearly 0 UL, 0 DL
[20:17:57] <The_8472> if the swarm is large enough those things should smooth themselves out over time
[20:18:09] <The_8472> well, in that case the seed is NOT fast enough ;)
[20:18:13] <Switeck> no, it gets WORSE as the swarm gets larger in my scenario
[20:18:32] <Switeck> to speed things up, I banned the seed
[20:18:58] <leotr> hello! i'd like to write pf rules to disallow traffic outside of country. Where can i read about bittorrent protocol?
[20:18:58] <The_8472> that doesn't make sense
[20:19:38] <Switeck> a seed that constantly uploads to everyone is slower than uploading constantly to everyone but me
[20:19:41] <Andrius> leotr, blocking peers based on location has nothing to do with bittorrent protocol
[20:19:48] <Switeck> all other things held constant
[20:20:14] <Andrius> leotr, anyway, here: http://bittorrent.org/beps/bep_0003.html http://wiki.theory.org/BitTorrentSpecification
[20:20:32] <The_8472> Switeck, well. i regularly observe torrents backed by about 6 100Mbit initial seeds and don't see any such behavior except for the very first few pieces
[20:20:37] <leotr> Andrius, i want to block only my bittorrent client. I don't want to block my browser
[20:20:51] <Switeck> well, that obviously isn't my scenario XD
[20:20:58] <The_8472> each peer downloads different pieces, after that they have enough to share to keep their asymmetric pipes busy for a while
[20:21:01] <Andrius> then it's a bittorrent client, not protocol issue
[20:21:24] <The_8472> well, then your initial seed is not fast enough to keep so many upload slots open
[20:21:29] <Switeck> correct!
[20:21:41] <The_8472> seed's fault. not really a problem with bittorrent
[20:21:44] <Switeck> but even upload slots = 10 is unreasonable for most
[20:22:01] <leotr> Andrius, i user uTorrent which supported ipfilter.dat. Is there a similar freebsd client. Seems like Transmission doesn't support it
[20:22:06] <The_8472> depends on the piece size
[20:22:08] <Switeck> most being "consumer" ISP lines (ADSL, cable, and low-end fiber optics)
[20:22:26] <Switeck> 3 KB/sec per upload slot is a reasonable rule-of-thumb.
[20:22:30] <Switeck> minimum
[20:22:38] <The_8472> i'd say  at least 4kB
[20:22:51] <The_8472> but really, it depends on the piece size too
[20:23:01] <The_8472> and you shouldn't seed popular torrents from home connections
[20:23:14] <Switeck> people tend to run more torrents at once than is reasonable, so some leeway has to be given in light of public torrent swarms (if that's where you're using these settings)
[20:23:37] <Switeck> there is no reason why a slow connection cannot seed a popular torrent
[20:23:48] <Switeck> and there should be decent settings to do so
[20:23:52] <The_8472> i'm talking about initial seeds
[20:23:57] <Switeck> I mean just that too
[20:24:06] <Switeck> 1 GB from 256kbit/sec upload line
[20:24:11] <The_8472> well, they're going to bore those people with 100Mbit down to death
[20:24:35] <Switeck> they can go download 100 other things at the same time
[20:24:48] <The_8472> if i'd release something i knew would be downloaded by more than a handful of people i'd throw it on a seedbox first
[20:24:58] <Switeck> BitTorrent should *NOT* cater exclusively to fast connections
[20:25:30] <The_8472> that has nothing to do with bittorrent
[20:25:40] <The_8472> it's more about a sane setup for content distribution
[20:25:52] <Switeck> I deem my scenario sane
[20:25:56] <Switeck> you just don't happen to like it XD
[20:26:15] <Andrius> leotr, I didn't need such feature, so I don't know. But there probably are clients that support something like that. Azureus/Vuze probably does, but I'm not sure
[20:26:16] <Switeck> now I don't claim it will be timely (meaning done in <2 days)
[20:26:47] <leotr> Andrius, thanks
[20:27:05] <The_8472> would be faster for me to drive to the university and upload with my lappy than seed something from here ^^
[20:27:45] <Switeck> I read the purpose of BitTorrent for typically small-scale distribution (<1000 peers) with limited bandwidth budgets, so that's why I came up with the hypothetical example.
[20:28:16] <The_8472> thing is, that doesn't happen in reality all that often
[20:28:23] <Switeck> I think it does
[20:28:33] <Switeck> it used to anyway XD
[20:28:39] <The_8472> well, maybe. but not for the stuff i download ^^
[20:28:53] <Switeck> seed boxes for the "common" folks used to be unheard of.
[20:29:13] <Switeck> and many places have *SHIT* internet
[20:29:47] <Andrius> or just *SHIT* ISP
[20:30:12] <The_8472> yes, but the most popular stuff is usually distributed by organized groups. who have at least 1 seedbox. if not even a whole network of those
[20:31:41] *** GTHK2 has joined #bittorrent
[20:32:29] <The_8472> but yes, if you want to seed from a home connection then some sane settings will be necessary
[20:32:32] <The_8472> but that's sortof obvious
[20:32:43] <Switeck> yeah, if you throw enough money at the problem it tends to go away XD
[20:33:10] *** GTHK2 has quit IRC
[20:33:14] <The_8472> well, a virtual root server doesn't cost much more than a decent internet connection here
[20:34:13] <Switeck> I'm >150 km from even another small city...server hosting here is a bit lacking.
[20:34:23] <The_8472> the difference is that you pay for your bandwidth instead of having a flatrate. but you get better upload speed too
[20:34:54] <Switeck> a single backhoe has on multiple occasions killed my internet connection
[20:35:09] <Switeck> and much of the area
[20:35:17] <The_8472> a what?
[20:35:24] <Switeck> trench digger
[20:35:31] <The_8472> ah
[20:36:19] <The_8472> well, that happens in cities too. but the damage is localized, usually fixed within hours and only happens once every few years
[20:36:44] <Switeck> the worst resulting outages for us lasted nearly 2 weeks and affected 1000's
[20:36:54] <The_8472> ouch
[20:37:07] <Switeck> I think internet was "free" that month :P
[20:37:18] <Switeck> (what little there was of it)
[20:37:48] <The_8472> odd how they can do so much digging in such a sparsely populated area
[20:38:01] <Switeck> there's over 50k people in my town
[20:38:18] <Switeck> it just happened the major highway to the south runs alongside the buried fiber lines
[20:38:29] <Switeck> and it's under widening roadwork...
[20:38:53] <The_8472> ah, and the left hand didn't know what the right hand did... i see
[20:39:16] <Switeck> highway department either didn't know exactly where the lines were at that point of the highway, or goofed
[20:39:46] <Switeck> being at the "end of the line" for the ISP means no redundancy. the redundant lines all ran through the same pipe.
[20:40:52] <Switeck> probably isn't more than a gbit/sec or 2 worth of bw all told
[20:41:51] <Switeck> 1st mile contention rato being over 40:1 does that :(
[20:41:58] <Switeck> or last mile
[20:43:12] <Switeck> For me, even perfect local peer discovery wouldn't help much.
[20:43:50] <Switeck> they'd likely give me <10 KB/sec (and me back to them)
[20:44:01] <Andrius> nice upload...
[20:44:24] <The_8472> unless it woudl be switched locally
[20:44:36] <Switeck> it's DOCSIS v1.1 cable
[20:44:40] <The_8472> rofl
[20:44:43] <The_8472> fail... cable
[20:45:01] <Switeck> not *JUST* cable, it's the older DOCSIS v1.1 cable.
[20:45:18] *** K`Tetch has quit IRC
[20:45:18] <Switeck> 9 mbit/sec upload bandwidth spread between ~100 cablemodems
[20:45:29] <The_8472> i know, but cable in itself is a big failure when it comes to internet connections. at least structurally
[20:45:50] *** K`Tetch has joined #bittorrent
[20:46:11] <Switeck> I agree that Cable cannot get a decent contention ratio
[20:46:29] <Switeck> not economically at least.
[20:46:41] <The_8472> DSL has other issues, but at least everyone has their own loop to the DSLAM
[20:46:50] <Switeck> and what it has is wasted
[20:47:27] <Switeck> many DSLAMs though might have 100-500 connected ADSL modems and a backhaul that's only 100 mbit/sec or OC-3.
[20:47:44] <Switeck> which doesn't move the bottleneck very far :(
[20:47:45] <The_8472> in the US maybe
[20:47:48] <The_8472> here it's a lot better
[20:47:56] <Switeck> or Canada, or Australia...
[20:48:03] <Switeck> or parts of the UK I've heard...
[20:48:17] *** K`Tetch has quit IRC
[20:48:49] <Switeck> You got VDSL2 in your area?
[20:49:04] <The_8472> haven't checked.
[20:49:31] <The_8472> probably still ADSL2+ (16 or 20Mbit max down)
[20:49:36] <Switeck> That beats cable and ADSL severely, but only for short <4km hauls
[20:49:41] <The_8472> VDSL(2) is mostly rolled out in urban centers
[20:49:52] *** K`Tetch has joined #bittorrent
[20:49:59] <The_8472> well, i'm less than 4km from the POP
[20:50:04] <Switeck> <- 6 mbit/sec DL cable
[20:50:25] <The_8472> heh, i'm 1Mbit down. i could upgrade though
[20:50:28] <Switeck> ADSL is cheaper only if I get 3 mbit/sec or less
[20:50:53] <The_8472> i.e. i'm not limited by the line (anymore), just by the contract
[20:50:53] <Switeck> and even then not by much due to phone line bundling.
[20:51:11] <The_8472> they're unbundled here :)
[20:51:13] <Switeck> what's the contention ratio?
[20:51:27] <Switeck> estimated and/or reported
[20:51:54] <The_8472> that i do not know, but a friend of mine has 20Mbit down 1.5Mbit up and uses it pretty much 24/7 without major drops
[20:52:11] <Switeck> probably 30:1 or less then for your friend
[20:52:14] <The_8472> so even if there's contention then there are enough people not using their pipes
[20:52:46] *** waldorf_ has joined #bittorrent
[20:52:58] <Switeck> your friend must have one of the updated ADSL2+ types...most can't break ~1.4 mbit/sec upload
[20:53:36] *** TheSHAD0W has quit IRC
[20:53:42] <The_8472> it's the fastest they offer anyway
[20:53:56] <Switeck> it might even be VDSL2?
[20:54:02] <The_8472> 50Mbit and more are only available where VDSL has been rolled out
[20:54:31] <The_8472> hrrm, let me check
[20:54:33] <Switeck> yeah, but on the upload side ADSL2+ suffers horribly.
[20:54:57] <The_8472> 25M down is their smallest VDSL packet
[20:55:22] <The_8472> so i think he's still on ADSL2+, since he got 20M
[20:57:26] <The_8472> no VDSL for me at least
[20:57:35] <The_8472> 16M is max for my number
[20:58:41] <Switeck> hopefully that doesn't feel limiting
[20:58:53] <Switeck> the upload probably would though.
[20:59:13] <The_8472> still would be more than what i have right now ^^
[20:59:30] <The_8472> but yeah. i'd rather get 4/4 than 16/1
[20:59:47] *** burris has quit IRC
[20:59:58] <The_8472> but SDSL is only available for business customers. for like 600?/month
[21:00:21] *** burris has joined #bittorrent
[21:01:16] <Switeck> that's insane!
[21:01:36] <The_8472> still cheaper than a dedicated line though
[21:01:40] <Andrius> I'm glad I live in a country with pretty good internet speed :) Bad thing is that some ISPs think that users don't need fast international traffic.
[21:01:44] <Switeck> I doubt a 100 mbit/sec seed box with 95 percentile would run that high
[21:01:58] <The_8472> true
[21:02:16] <Switeck> Andrius, that Israel?
[21:02:33] <Andrius> Lithuania :)
[21:02:36] <Switeck> ah
[21:03:02] <The_8472> it's the same in india or australia too
[21:03:10] <The_8472> and south korea i've heard
[21:03:13] <Switeck> Lots of Eastern Europe posters at uTorrent forums talk about tiered internet connections -- often >10 mbit/sec locally but <2 mbit/sec international
[21:03:45] <The_8472> ISP-local multicast would greatly help in that case
[21:03:53] <Switeck> and no good way to configure a client to utilize that disparity
[21:04:04] <Andrius> Switeck, yeah, 100mbit local lines aren't that uncommon, while international traffic is like 2mbps
[21:04:06] <Switeck> it's not just ISP, it's in-country bandwidth
[21:04:14] <The_8472> that's even worse
[21:04:29] <The_8472> because the internet doesn't make a distinction of which IP belongs to "your country" and which doesn't
[21:04:53] <Switeck> exactly, the AS numbers could differ wildly
[21:05:44] <The_8472> though bittorrent should be able to find faster peers simply at random. it takes time though
[21:05:56] <Andrius> still, it's better than slow connection everywhere :)
[21:06:02] <Switeck> it's configuring your upload for that which is the kicker
[21:06:18] <Switeck> web browsing can be harsh as a result
[21:06:27] <Andrius> The_8472, I still limited my upload to lower limit (international one), wasting ton of upload for local peers
[21:06:49] <Switeck> if you could create a local ip list, you could run a special separate BT client for just that
[21:06:54] <Andrius> now I have good ISP, so it's not an issue, but it was earlier
[21:07:11] <The_8472> Switeck, that's probably the easiest approach
[21:07:15] *** TheSHAD0W has joined #bittorrent
[21:07:37] <Andrius> yeah, had that idea. I think I even used it combined with local private tracker
[21:07:59] <The_8472> but then you'd still need a good list of ASNs or IP-ranges
[21:08:58] <Andrius> well considering there aren't that many "local" peers, it's too much work and doesn't even guarantee I'll find any peers
[21:09:18] <Andrius> maybe in larger countries it would make more sense
[21:09:29] <Switeck> AND...they could already be connected to the 'international' BT client
[21:09:41] <Switeck> which might then "same ip" disconnect your local BT client
[21:10:10] <The_8472> just ban the national ones on your international client
[21:10:14] <Andrius> well downloading the same torrent in two clients isn't a very good idea :)
[21:10:20] <Switeck> a user-definable ip range table for "LAN" connections in Azureus would work
[21:10:34] <The_8472> Andrius, it's ok if they're connected to each other via lan discovery
[21:10:42] <Switeck> which seems more intelligent than hard coded there anyway XD
[21:10:49] <Andrius> didn't think about it
[21:11:04] <The_8472> Switeck, it's user-defineable actually
[21:11:24] <Switeck> then that is the best way to do it
[21:11:37] <The_8472> just not meant for entering bulk amounts of data. ^^
[21:11:46] <Switeck> scale it XD
[21:12:23] <Switeck> I would hope the range overlaps would keep the total number of ranges below 1000.
[21:12:34] <Switeck> possibly below 100 for the busiest parts
[21:12:44] <The_8472> i think a 3MB long string is the max we would persist
[21:13:24] <The_8472> as things are right now
[21:13:39] <Switeck> bare IPv4 ips, that's overkill
[21:13:52] <Switeck> problem is...IPv6
[21:14:05] <The_8472> that's why we use strings that get parsed ^^
[21:14:12] <The_8472> like with other IP (block) lists
[21:14:28] <Switeck> I meant IPv6 ranges...uTorrent can't handle that at all XD
[21:14:44] <The_8472> heh, not sure if we handle them either
[21:14:49] <The_8472> that hasn't come up yet
[21:15:14] <Switeck> ok, it just did...for finding local IPv6's XD
[21:15:48] <The_8472> there isn't even geoIP for v6, is there?
[21:16:55] <Switeck> like everything is their own little LAN IPv6 ranges atm?
[21:17:17] <The_8472> hrm?
[21:17:49] <Switeck> I mean the IPv6 addresses aren't guaranteed globally unique and/or locatable?
[21:18:06] <The_8472> oh, they are. except the fe80: stuff
[21:18:19] <leotr> how does upload work when incoming connections are closed on firewall?
[21:18:31] <The_8472> TCP is bidirectional
[21:18:39] <Switeck> upload doesn't care HOW the connection is made
[21:18:46] <Switeck> just that a connection exists
[21:19:00] <The_8472> in other words: just like webbrowsing works
[21:19:18] <Switeck> even firewalled, you upload a bit to webpages
[21:19:37] <cyb2063> as in: the other end is not firewalled.
[21:19:52] <Switeck> obviously 1 end has to be (effectively) unfirewalled
[21:20:15] <Switeck> otherwise they would never connect to each other in the first place.
[21:20:21] <leotr> ahh, does that means if port is closed then i can ask if someone needs some piece of file and then give it
[21:20:41] <Switeck> in theory, yes. in practice it won't work if they're firewalled too
[21:20:57] <The_8472> uhh... you lack basic understanding how p2p and the internet in general works i think
[21:22:00] <leotr> can 2 firewalled users exchange between each other?
[21:22:09] <DWKnight> never directly
[21:22:11] <The_8472> not in the general case, no
[21:22:23] <DWKnight> you need a way to fake unfirewalled
[21:22:32] <DWKnight> or get genuine unfirewalled
[21:22:35] <DWKnight> on either end
[21:23:20] <The_8472> bittorrent works for a firewalled peer, due to other people running unfirewalled ones. but it generally works better if you're unfirewalled too
[21:23:32] <The_8472> because other peers (including firewalled ones) can connect to you too
[21:24:08] *** Pollo12 has joined #bittorrent
[21:24:46] *** Pollo12 has left #bittorrent
[21:25:06] <Switeck> on a private tracker, you'll likely get banned fast if you're always firewalled.
[21:25:35] <The_8472> hehe, too bad for those behind one of those stupid ISP-level NATs
[21:26:15] <Switeck> wireless and satellite ISPs especially
[21:26:45] <leotr> A is firewalled and has piece number N, B is unfirewalled and doesn't have that piece, C is firewalled and also has that piece. Then C has to wait until B has that piece. B will get that piece only if A connects to B. Am I right?
[21:27:03] <Switeck> no
[21:27:17] <Switeck> A can connect directly to C
[21:27:22] <Switeck> my bad
[21:27:24] <Switeck> A to B
[21:27:35] <Switeck> C can connect to B as well
[21:27:49] <leotr> Then C has to wait until B has that piece
[21:27:59] <cyb2063> yes
[21:28:01] <Switeck> A cannot connect to C, C cannot connect to A, B cannot connect to A OR C
[21:28:01] <The_8472> C already has that piece
[21:29:00] <The_8472> you said C also has that piece
[21:29:04] <The_8472> thus C does not have to wait for B
[21:29:10] <Switeck> If you meant both B and C doesn't have piece "N", then B first has to get the piece (hash checked and all) before C can start getting that piece.
[21:29:30] <leotr> yes i meant that
[21:29:39] <Switeck> worst-case might be a single unfirewalled seed with 10 firewalled peers
[21:29:59] <The_8472> then you're back to classic client-server ^^
[21:30:12] <Switeck> the seed has to upload the whole torrent 10 times for them all to complete the torrent.
[21:30:16] <leotr> then does that all mean that firewalled peers cannot create torrents?
[21:30:31] <DWKnight> no
[21:30:34] <Switeck> a firewalled seed could work if every peer is unfirewalled
[21:30:35] <The_8472> they can, but they're a lot less effieicent at it
[21:30:39] <DWKnight> it means they can't distribute them efficently
[21:30:41] <Switeck> or most peers are unfirewalled
[21:31:11] <Switeck> it might even be "useful" as an initial seed to make the torrent spread faster...
[21:32:09] <Switeck> by eliminating firewalled peers, the peer pool could still be "big enough" to upload to everyone else.
[21:32:33] <Switeck> the seed might need help finding peers initially though :(
[21:34:26] <leotr> Marks wants to give file windows7.iso (a useful one). But people except John on torrent tracker are linux users and hate windows. Mark and Joht are firewalled, others are not. Does that mean that John will never obtain that iso file?
[21:34:55] <The_8472> correct
[21:35:26] <The_8472> of course you could just have asked if other pers on a tracker sharing different torrents have anything to do with your torrent
[21:36:06] <The_8472> well. unless those windows-haters download windows to have windows-burning parties
[21:36:12] <The_8472> in that case they can download it
[21:36:28] <leotr> :)
[21:36:49] <Switeck> at least in my experience, people on the same tracker are unaware of each other... outside of private trackers
[21:37:04] <The_8472> most of the time they're even unaware of the tracker
[21:37:16] <The_8472> i've seen many people call mininova a "tracker site"
[21:37:23] <Switeck> "on" the tracker is technically only true for a couple seconds out of an hour.
[21:37:44] <The_8472> or never, if the tracker is down and you're getting peers via DHT/PEX
[21:37:51] <Switeck> that also
[21:38:20] <Switeck> and there's many "lite" trackers now which don't require uploading the .torrent file to them
[21:38:35] <Switeck> so even having a strong concept of what files are hosted by the tracker is difficult.
[21:39:02] <Switeck> (.torrent files, not the contents of the torrents.)
[21:39:40] <Switeck> I've seen misnamed .torrent files as well -- loading them into my BT client causes them to say they're something else. :P
[21:39:41] <cyb2063> for me a tracker is only tracking, not hosting torrent files.
[21:40:07] <The_8472> indexing site: hosts and lists .torrent files; tracker: tracks peer/seed lists for a torrent
[21:40:12] <The_8472> they're unrelated
[21:40:16] <cyb2063> yep
[21:40:56] <leotr> Finally, i can write pf rule that will disallow connections to my bittorrent client from ip addresses outside of my country, but i cannot forbint my bittorrent client to connect to peers outside of my country because ports are randomized :|
[21:41:13] <leotr> s/forbint/forbid
[21:41:22] <The_8472> bind your client to a source port
[21:41:26] <The_8472> and filter based on that
[21:41:34] <Switeck> or source port range?
[21:41:39] <The_8472> one is enough
[21:42:07] <The_8472> source and listening port even can be the same
[21:42:21] <Switeck> if you're getting 5 torrents from the same series and 1 peer is on all 5, you might experience minor issues
[21:42:29] <The_8472> well, yes
[21:42:40] <Switeck> I know, because I've done exactly that
[21:42:41] <The_8472> but you shouldn't be downloading them all at once anyway
[21:42:50] <Switeck> yeah I should XD
[21:42:52] <The_8472> since they'll probably have a mostly congruent set of peers
[21:43:01] <DWKnight> [4:42:44pm] <The_8472> but you shouldn't be downloading them all at once anyway <-- there are a lot of things that people shouldn't do but do anyway
[21:43:03] <Switeck> they were tiny files individually
[21:43:06] <DWKnight> that included
[21:43:16] <Switeck> and uploading was very intermittent with them afterwards
[21:43:49] <Switeck> If I want to start 5 torrents at once, my linespeed is fast enough it's not an issue.
[21:44:17] <The_8472> if they are hosted by the same peers then it's a waste of bandwidth
[21:44:22] <The_8472> by splitting their bandwidth over 5 torrent
[21:44:27] <The_8472> s
[21:45:04] <leotr> how to bind client to source port?
[21:45:11] <The_8472> that is client-specific
[21:45:22] <The_8472> some option/setting
[21:45:23] <leotr> any Transmission users here?
[21:46:09] <leotr> or does someone know uTorrent analogue for freebsd?
[21:46:25] <Switeck> even if many of the peers are on the same 5 torrents, it's not an issue
[21:46:36] <leotr> uTorrent has ip filtering capabilities.
[21:47:38] <The_8472> idk, Azureus might run on freebsd, but it's not one of our supported systems
[21:47:41] <Switeck> Those peers were likely uploading at their max or nearly idle before I arrived.
[21:48:09] <The_8472> but some other clients have ip block lists too
[21:48:34] <Switeck> and they may be using junk like Peer Guardian 2 as well
[21:48:49] <Switeck> for *TWICE* the protection! XD
[21:49:37] <leotr> Azureus is written in Java, i'd like something lightweight
[21:50:19] <The_8472> yeah, well... most of the linux clients can probably run on freebsd too with some tinkering
[21:50:55] <leotr> of maybe i should run uTorrent under wine ^)
[21:51:02] <leotr> or*
[21:51:18] <Switeck> if you already have the latest dev version of wine, sure
[21:51:27] <Switeck> I don't recommend uTorrent with wine 1.0
[21:52:00] <leotr> ok, thank you all. I was very pleased, bye
[21:52:07] <DWKnight> most of uT's support staff won't even talk to you for more than 30 seconds if you aren't using the latest dev version of wine
[21:52:07] <cyb2063> The_8472: Vuze runs just fine on FreeBSD here. :)
[21:52:19] <The_8472> cyb2063, it does? i'm surprised ^^
[21:52:28] <cyb2063> for a long time
[21:52:49] <The_8472> ah right, there was someone who was adventurous enough
[21:52:59] *** leotr has left #bittorrent
[21:53:09] <The_8472> still, it doesn't run out of the box, does it?
[21:53:22] <cyb2063> more or less it does.
[21:53:48] <The_8472> there must be something wrong then *scratching head*
[21:53:54] <The_8472> things aren't supposed to "just work"
[21:54:49] <cyb2063> there are a few patches http://www.freebsd.org/cgi/cvsweb.cgi/ports/net-p2p/vuze/files/
[21:56:03] <cyb2063> and since parg fixed some java specific problems some of them are not needed any more.
[21:56:13] <The_8472> ah, ok
[22:00:01] <cyb2063> funny when people say vuze/java is too bloated, but then want to run a client with wine.
[22:00:19] <DWKnight> the java vm is a bit heavier than wine overall
[22:01:49] <cyb2063> yes, but still I think they should find a native client, so there is no need for wine at all.
[22:03:18] <kjetilho> uh, running an EXE in Wine has less overhead than running a Python client
[22:03:39] <kjetilho> although *I* run a Python client since overhead isn't an issue anyway :)
[22:04:00] <cyb2063> really? uh.
[22:04:14] <cyb2063> the exe part, not you running a python client.
[22:04:47] <kjetilho> compiled code is good
[22:04:48] <cyb2063> ah well, if it works and makes them happy. why not.
[22:05:20] <kjetilho> cyb2063: it's sad, but Firefox under Wine feels snappier than native Firefox
[22:06:26] <kjetilho> the UI libraries clearly are better optimised, both by Firefox crew to run well on WIN32 API and by Wine developers emulating the WIN32 API.
[22:08:57] *** kmels-away has quit IRC
[22:09:07] *** kmels-away has joined #bittorrent
[22:09:07] *** kmels-away is now known as kmels
[22:09:47] *** kmels has left #bittorrent
[22:26:25] *** leotr has joined #bittorrent
[22:27:26] <leotr> it's me again. Finally i could find a good bittorrent client- KTorrent. It has similar interface to uTorrent and suppots ip blocking ^)
[22:30:29] <cyb2063> are you using kde?
[22:31:58] <K`Tetch> why do you need IP blocking?
[22:32:25] <K`Tetch> does no damned good
[22:51:53] <leotr> K`Tetch, i'm blocking IPs out of my country because traffic is limited to 10Gb and unlimited inside of my country
[22:52:20] <K`Tetch> ah
[22:53:26] *** leotr has quit IRC
[22:53:30] *** _rafi_ has quit IRC
[22:54:15] <DWKnight> that's on the shortlist of why you'd actually IP block
[22:54:18] <DWKnight> and have it work
[23:26:30] <K`Tetch> mostly
[23:27:32] <DWKnight> it's far more effective than the more common reason to use ip block
[23:27:51] *** cyb2063 has quit IRC
[23:28:59] <Andrius> which one, "he's not uploading to me" or "I'll block random ranges of IPs and be safe"?
[23:30:29] <DWKnight> latter
[23:34:46] *** Miller` has joined #bittorrent
[23:47:11] *** rrr_ has joined #bittorrent
[23:53:24] *** KyleK_ has quit IRC





top