[00:00:33] <The_8472> next step... a tree of averages
[00:01:32] <choykloun> so with 1 little hack we could publish full pubkeys transparently
[00:02:24] <The_8472> uhm... that screws up routing tables since only 1 ID per IP is allowed
[00:02:36] <choykloun> second ipaddr can be a hash of first
[00:02:38] <The_8472> and they're expected to be more or less constant
[00:02:48] <choykloun> with first byte set to lie multicast network
[00:03:20] <The_8472> not sure what you're getting at.
[00:03:29] <choykloun> like*
[00:03:31] <choykloun> well
[00:03:38] <choykloun> it wouldnt appear to normal clients as the same node
[00:03:45] <choykloun> just as a client with a crazy-ass ip addr
[00:03:53] <choykloun> or different port on same ipaddr maybe?
[00:04:06] <choykloun> you know much more about reallife dht than me so
[00:04:23] <The_8472> what do you want to do actually xD
[00:04:30] <choykloun> publish the 32 byte pubkeys
[00:04:35] <choykloun> using nodeids
[00:04:53] <The_8472> publish... how would you associate them with the IP/Port?
[00:05:16] <choykloun> like 2 parts
[00:05:21] <choykloun> first is the 160 bit nodeid as seen from dht
[00:05:26] <choykloun> has real ipaddr and port
[00:05:48] <choykloun> second is the remaining 96 bits + something relating it to the first
[00:05:53] <choykloun> so they can be linked regardless of order
[00:06:09] <The_8472> nodes wouldn't insert that into their routing tables or forward that to other nodes
[00:06:14] <choykloun> k
[00:06:31] <The_8472> you only get into routing tables if you're reachable. and even then it requires luck
[00:06:34] <choykloun> not even with same first 32 bits and same ipaddr
[00:06:44] <choykloun> you'd be reachable, on a different port
[00:07:04] <The_8472> you only get inserted into routing tables for sure if you're a neighbor of the node you want to join
[00:07:13] <The_8472> close to the root you're just randomly chosen
[00:07:22] <The_8472> your 2nd port wouldn't get into the bucket too
[00:07:39] <choykloun> k k
[00:07:40] <choykloun> :(
[00:07:43] <The_8472> we could of course extend node IDs to 32 bytes
[00:07:50] <The_8472> but that would completely break compatibility
[00:08:54] <choykloun> haha ya
[00:09:12] <choykloun> so then remains an extension of the protocol
[00:09:27] *** teamcoltra has joined #bittorrent
[00:12:58] <choykloun> you could mark nodes you have exchanged keys with as trusted
[00:13:08] <choykloun> which would make you trust pubkeys from them
[00:13:08] <choykloun> etc
[00:13:13] <choykloun> pgp web of trust style
[00:14:26] <choykloun> anyway
[00:14:35] <choykloun> as for KEX pubkey 32 bytes extra isnt really bad
[00:14:42] <choykloun> not worth spending any effort on avoiding
[00:16:17] <choykloun> as for protection against mitm
[00:16:22] <choykloun> lets start by adding clientside puzzle to it
[00:17:02] <choykloun> my old util keyzah to prevent bruteforce of cryptoloop passphrases could come in handy
[00:17:11] <choykloun> but its quite memory hungry
[00:17:52] <choykloun> well considering at least tens of thousands of users will be doing this even 512k would be an issue for the sniffers
[00:22:42] *** DWKnight has joined #bittorrent
[00:32:42] <choykloun> n Log: Got ping query response from 194.71.126.18:4242
[00:32:44] <choykloun> n Log: Storing node (nodeid 0x3c4ad726, from 194.71.126.18:4242)
[00:32:47] <choykloun> n Log: Keys exchanged - agreed shared secret id 0x4b385894, peer public key id 0xaf2924df
[00:41:04] <choykloun> and with multiple nodes
[00:41:07] <choykloun> n Log: Got ping query response from 172.23.0.2:4242
[00:41:07] <choykloun> n Log: Storing node (nodeid 0x1ae5409c, from 172.23.0.2:4242)
[00:41:07] <choykloun> n Log: Keys exchanged - agreed shared secret id 0xd49327ea, peer public key id 0xf7cdc51a
[00:41:11] <choykloun> n Log: Got ping query response from 88.80.20.41:4242
[00:41:13] <choykloun> n Log: Storing node (nodeid 0x9296d7b9, from 88.80.20.41:4242)
[00:41:16] <choykloun> n Log: Keys exchanged - agreed shared secret id 0x35253132, peer public key id 0x6257dc6c
[00:41:19] <choykloun> \o o/
[00:41:44] *** Snoopotic has quit IRC
[00:44:48] <choykloun> now lets agree on the damn basics of packet encryption so that people can experiment with it
[00:45:39] <choykloun> by the way
[00:46:02] <choykloun> isnt the token arbitrary length
[00:47:27] <The_8472> it is, sortof
[00:48:27] *** GTHK has joined #bittorrent
[00:48:28] <choykloun> transid certainly isnt
[00:48:51] <The_8472> well, in my implementation it is, idk about others
[00:49:10] <choykloun> i fucking crashed router.bittorrent.com with a weird one
[00:49:36] <choykloun> and with a like 16 byte one a lot of clients seems to throw it away
[01:01:43] *** andar has quit IRC
[01:03:39] *** andar has joined #bittorrent
[01:04:11] *** Andrius has quit IRC
[01:13:39] *** _rafi2_ has joined #bittorrent
[01:15:09] <choykloun> anyways
[01:15:17] <choykloun> will design a proposed protocol for crypto
[01:15:22] <choykloun> lets see if you like it or not
[01:19:05] *** _rafi_ has quit IRC
[01:23:09] *** Waldorf has quit IRC
[01:33:27] <The_8472> WOW!
[01:33:40] <The_8472> bloom filters are fscking awesome...
[01:33:59] <choykloun> i do recall it from some while long ago
[01:34:20] <choykloun> it gives you false negatives but never false positives right
[01:34:23] <choykloun> or was it the other way around
[01:34:29] <choykloun> ah
[01:34:32] <choykloun> false pos says wiki
[01:34:40] <The_8472> yep, but that's not the thing
[01:35:04] *** mxs_ has joined #bittorrent
[01:35:07] <choykloun> ya high performance low memory use etc
[01:35:08] *** mxs has quit IRC
[01:35:12] <The_8472> i'm generating 10k peers randomly, sampling them randomly into 8 bloomfilters
[01:35:12] *** mxs_ is now known as mxs
[01:35:22] <The_8472> then i run them through the tree-average i've devised
[01:35:37] <choykloun> the mesh p2p thingie i designed in like 2000 was pretty nice though
[01:35:39] *** bittwist has joined #BitTorrent
[01:35:47] <choykloun> much less routing issues than dht
[01:35:47] <The_8472> and it returns the number of unique values i've inserted across all 8 filters pretty accurately
[01:36:02] <choykloun> though it was designed for transfering messages
[01:36:07] <choykloun> not doing a dht
[01:36:33] <choykloun> routing was very simple
[01:36:51] <choykloun> the complex part was the algorithm that determines the addr that a new node should be assigned
[01:37:17] <choykloun> coz knowing the addrs of your neighbors and the target a node could always route a msg correctly
[01:38:06] <The_8472> 8 filters, inserting 50 to 550 each out of 550 possible values (thus the contents of each 8 filters is almost identical) -> it reports 564 unique values
[01:38:57] <The_8472> 8 filters, inserting 50 to 550 each out of 15000 possible values (overall 2157 unique values inserted) -> it reports 2179 unique values
[01:39:00] <choykloun> fuck i so feel like writing a 100% asm benc decoder just to tease you guys :)
[01:39:23] <The_8472> you like to torture yourself, don't you?
[01:39:28] <choykloun> its not torture!
[01:39:35] <choykloun> you should see my 3k line asm project
[01:39:41] <choykloun> highly experimental
[01:39:54] <The_8472> i think there is some desktop OS written in ASM
[01:39:57] <choykloun> so parts of it makes people question my sanity
[01:40:00] <choykloun> ya
[01:40:02] <choykloun> MSDOS? :P
[01:40:06] <choykloun> but i know which one you are talking about
[01:40:10] <choykloun> some finnish dude
[01:41:05] <The_8472> http://en.wikipedia.org/wiki/MenuetOS <- there
[01:41:10] <choykloun> ya exactly that one
[01:41:17] <choykloun> in the good ol days all os were 100% asm
[01:41:21] <choykloun> ibm os/360 is 1.9 million lines
[01:41:28] <choykloun> still havent succeeded in compiling all of it :(
[01:41:38] <choykloun> i want to get TSO up and running !!!
[01:52:07] <The_8472> yus! now i have everything i need for a DHT scrape spec.
[01:53:16] <choykloun> haha, my name looks so funny in khmer
[02:38:52] *** _rafi2_ has quit IRC
[02:45:43] <swolchok> oceanstore used bloom filters for something or other
[02:45:53] <swolchok> I think it was based on tapestry though. definitely not kademlia.
[02:54:31] *** stalled has quit IRC
[02:57:28] <choykloun> my dht testing setup is so pathetic
[02:57:42] <choykloun> the clients dont even consider any nodes worth announcing to
[03:04:45] <choykloun> s 1261793595 Node 75.40.227.109:13549 distance 0x00264fae best 0x00000138 worst 0xf7bc2aee nodeID C9377FA55F22D9016F3F11E0CD4C0FF01A14CD2900
[03:04:48] <choykloun> s 1261793595 Node 220.141.184.121:13228 distance 0xa8b03dbf best 0x00000138 worst 0xf7bc2aee nodeID 61A10DB4B0D642D64372B00879A6D1A7D04934DD00
[03:04:52] <choykloun> hm
[03:04:52] <choykloun> my impl could be behaving worse i guess
[03:05:25] <choykloun> but some of my algos would probably give you guys a heart attack
[03:07:28] <choykloun> dhtlog_c:c 1261793595 Prospective announce node - sending get_peers to 72.71.9.172:50837 distance 0x0094eb84 nodeID C985DB8FD37597FD31A753FD1953F70284216F43
[03:07:31] <choykloun> dhtlog_c:c 1261793595 Announcing to 193.238.92.38:62065 distance 0x0095ab25 nodeID C9849B2E821281B9BEE8256F63635EC442004E98 currentToken 4D882043C06B02FA1B32DB960B77DFA5D040EABF
[03:07:35] <choykloun> dhtlog_c:c 1261793595 Announcing to 85.228.129.108:29693 distance 0x0098489f nodeID C9897894EECDB440CD834C1ED5DCBCDD30BCB722 currentToken BA5FF00A902356CF5471178963886D6023212064
[03:07:40] <choykloun> dhtlog_c:c 1261793595 Announcing to 60.241.224.134:48583 distance 0x009c0219 nodeID C98D3212326667AC550711AC7076B3076FE4D66B currentToken 52F2FAD381A59DB1B731267F6756BCC8122223B4
[03:07:44] <choykloun> ...
[03:07:47] <choykloun> dhtlog_c:c 1261793595 Got announce_peer query response from 114.154.202.131:16213
[03:07:50] <choykloun> dhtlog_c:c 1261793595 Got announce_peer query response from 121.95.119.95:20904
[03:07:54] <choykloun> dhtlog_c:c 1261793595 Got announce_peer query response from 123.130.158.129:14244
[03:08:36] <choykloun> hm should try using all bits instead of just the most significant 32 when deciding
[03:08:51] <swolchok> why do they all have IDs close together?
[03:08:59] <swolchok> at least pick 'em randomly
[03:09:08] <choykloun> hm i think you misunderstand
[03:09:17] <choykloun> they are supposed to have ids close together, thats what the code looks for :P
[03:09:25] <choykloun> for the proper nodes to announce a spcific info_hash to
[03:09:25] <swolchok> oh
[03:09:28] <choykloun> :P
[03:09:53] <choykloun> my nodeid's all look like this
[03:09:56] <choykloun> #define LOCAL_DHT_ID "\x3c\x4a\xd7\x26\x95\x31\xce\x2c" "ESTOYKH.COM"
[03:10:05] <choykloun> (so people know where to complain when i break stuff :PPP)
[03:10:52] <choykloun> the code also supports doing crazy shit like having the same node listen on several different addr:port with different nodeid's for each
[03:13:17] <choykloun> but normal behavior is set once at compile time :P
[03:38:26] *** kwinz2 has joined #bittorrent
[03:50:42] <TheSHAD0W> choykloun: Uh, where exactly are they supposed to complain?  Your website is a bit...  abstract?
[03:57:58] *** The_8472 has quit IRC
[03:58:19] *** wadim has joined #bittorrent
[03:58:21] *** wadim is now known as The_8472
[04:02:45] *** init0 has quit IRC
[04:05:59] *** init0 has joined #bittorrent
[04:06:11] <choykloun> it has a mail addr?
[04:06:18] <choykloun> right under the list of ecodes!
[04:07:04] <choykloun> and its not abstract, its very concrete! from the comments to one of the first preemptive multitasking implementations anywhere!
[04:07:18] <choykloun> imagine the challenges of implementing a scheduler in the 70's
[04:07:27] <choykloun> didnt exactly have any papers on the subject
[04:07:52] <choykloun> there was probably a lot of philosophers eating spaghetti before they got everything functional
[04:09:48] <choykloun> its from TSO in os/360
[04:09:53] <choykloun> such a weird system
[04:10:04] <choykloun> multitasking is an add-on component!
[04:10:06] <choykloun> ?!?!?!
[04:10:29] <choykloun> and you install it using JCL scripts on punched cards and source code on tape
[04:12:10] <choykloun> havent gotten os/360 to compile yet :(
[04:12:17] <choykloun> got the basic build system up but havent gotten past HASP :(
[04:12:55] <choykloun> 1.9 million lines of asm in total
[04:14:01] <choykloun> and i finally understand how computer illiterate ppl must feel
[04:14:02] <choykloun> HASP is to MVT what JES2 is to later operating systems: a job entry subsystem. It manages SYSIN and SYSOUT, keeps a queue of jobs ready to begin execution, and starts them according to a priority scheme. It makes running MVT much simpler and more efficient. It was quite rare to find an MVT system that did not include HASP.
[04:20:18] <choykloun> http://thefin.knark.net/memoryhole/user/os360ipl.png <== os360 after booting (IPLing in mainframe jargon)
[05:23:24] *** Switeck has quit IRC
[06:02:17] *** GTHK has quit IRC
[06:16:01] *** K`Tetch has quit IRC
[06:40:38] <choykloun> hey why are some clients so aggressive in connecting
[06:40:54] <choykloun> 06:50:02.877023 IP (tos 0x0, ttl 114, id 27611, offset 0, flags [DF], proto TCP (6), length 48) 76.73.8.10.4710 > 88.80.20.41.8941: S, cksum 0xfc4a (correct), 3565141045:3565141045(0) win 65535 <mss 1460,nop,nop,sackOK>
[06:40:59] <choykloun> 06:50:03.533296 IP (tos 0x0, ttl 114, id 29913, offset 0, flags [DF], proto TCP (6), length 48) 76.73.8.10.4710 > 88.80.20.41.8941: S, cksum 0xfc4a (correct), 3565141045:3565141045(0) win 65535 <mss 1460,nop,nop,sackOK>
[06:41:03] <choykloun> 06:51:07.294976 IP (tos 0x0, ttl 114, id 2328, offset 0, flags [DF], proto TCP (6), length 48) 76.73.8.10.2273 > 88.80.20.41.8941: S, cksum 0x944e (correct), 2072741547:2072741547(0) win 65535 <mss 1460,nop,nop,sackOK>
[06:41:09] <choykloun> etc
[06:41:11] <choykloun> like 20 attempts
[06:49:56] *** andar has quit IRC
[06:50:50] *** andar has joined #bittorrent
[06:57:08] *** _rafi_ has joined #bittorrent
[07:07:05] *** andar has quit IRC
[07:07:15] *** andar has joined #bittorrent
[08:44:29] *** chelz has joined #bittorrent
[08:48:48] *** chelz has quit IRC
[08:57:52] <choykloun> n Log: Failed decoding bencoded UDP packet from 85.177.166.23:44103 (key is not a string)
[08:57:58] <choykloun> 72 5A D5 44 05 10 B2 2B 86 32 69 19 84 1B CC 22 5D 60 A6 DF 24 2A 88 6F rZ.D...+.2i...."]`..$*.o
[08:58:02] <choykloun> 56 13 EC 7A C5 53 0D 59 30 3F C7 AA CA 34 FC D3 B2 25 13 1E A5 E1 80 BB V..z.S.Y0?...4...%......
[08:58:05] <choykloun> E5 21 65 83 18 59 1B B1 F6 FE 9D ED 2C 08 D4 87 B8 23 D3 6C D3 1B BE 16 .!e..Y......,....#.l....
[08:58:09] <choykloun> 8E 8E 24 CA 48 75 33 BE 5E BC 30 B9 53 31 A2 76 54 54 BE 00 A6 B9 FB C6 ..$.Hu3.^.0.S1.vTT......
[08:58:12] <choykloun> 1F 3D 0A DF A8 92 A9 0E 47 BB DA A9 29 AA 03 22 50 4C 19 47 B3 9B 57 6C .=......G...).."PL.G..Wl
[08:58:16] <choykloun> D0 90 E8 18 FE 7D E1 C0 F6 8D AA 79 35 91 4F 01 .....}.....y5.O.
[08:58:18] <choykloun> huh
[08:58:21] <choykloun> why the hell did i get that packet
[09:15:21] *** bt42 has joined #BitTorrent
[09:24:56] *** stalled has joined #bittorrent
[09:35:40] *** bittwist has quit IRC
[09:39:30] *** Waldorf has joined #bittorrent
[09:42:07] <choykloun> 118164 known nodes and counting...
[09:52:10] *** Andrius has joined #bittorrent
[10:15:49] <choykloun> 263525  known nodes
[10:24:20] *** htunk has joined #bittorrent
[10:32:30] *** bittwist has joined #BitTorrent
[10:43:59] *** andar_ has joined #bittorrent
[10:44:20] *** andar_ has quit IRC
[10:52:31] *** bt42 has quit IRC
[11:29:21] *** goussx has quit IRC
[11:29:35] *** goussx has joined #bittorrent
[11:30:47] *** bt42 has joined #BitTorrent
[11:51:12] *** bittwist has quit IRC
[12:29:48] *** ivan` has quit IRC
[12:29:59] *** ivan` has joined #bittorrent
[13:21:17] *** The_8472 has quit IRC
[13:24:36] *** kwinz2 has quit IRC
[13:29:39] *** goussx has quit IRC
[13:32:39] *** The_8472 has joined #bittorrent
[13:36:05] *** goussx has joined #bittorrent
[13:57:02] *** n215 has joined #bittorrent
[13:57:51] <n215> can opentracker run  on openbsd/freebsd ?
[14:01:26] <The_8472> since there's a libowfat for bsd i would say yes
[14:02:13] *** bittwist has joined #BitTorrent
[14:02:43] <n215> thanks
[14:03:51] *** kwinz2 has joined #bittorrent
[14:04:58] <alus> libdjb > libowfat
[14:05:39] <alus> why would you re-release it under a new name with a more restrictive license...
[14:07:05] <The_8472> http://www.fefe.de/libowfat/ <- reasons stated here
[14:08:54] <alus> "because Debian said so"?
[14:08:56] <alus> lame
[14:10:05] *** bt42 has quit IRC
[14:37:25] *** kwinz2 has quit IRC
[14:41:44] *** kwinz2 has joined #bittorrent
[15:14:06] *** echelog has joined #bittorrent
[15:19:31] *** bt42 has joined #BitTorrent
[15:27:07] *** bittwist has quit IRC
[15:39:38] <Astro> I see you're all having much fun with the DHT?
[15:39:47] <Astro> I'm going to do a lightning talk about it on 26c3
[15:40:00] <Astro> don't worry, I'm preparing to write a paper on it soon
[15:53:04] *** kwinz2 has joined #bittorrent
[16:06:19] *** _rafi_ has quit IRC
[16:06:22] <The_8472> are you only going to tease or do you have some tidbits of information for us? ^^
[16:07:42] *** GTHK has joined #bittorrent
[16:21:56] *** bittwist has joined #BitTorrent
[16:29:48] *** bt42 has quit IRC
[16:37:04] <choykloun> so, can we agree on the basic packet encryption format sometime this year? :P
[16:37:36] <DWKnight> considering how little is left with "this year" maybe not
[16:45:04] <choykloun> kinda fascinating that i see over 400k dht nodes
[16:45:07] <choykloun> now thats some networking
[16:45:38] *** kwinz2 has quit IRC
[17:10:33] *** Gottaname has joined #bittorrent
[17:20:48] *** The_8472 has quit IRC
[17:21:09] *** wadim has joined #bittorrent
[17:21:10] *** wadim is now known as The_8472
[17:24:09] *** Gottaname has quit IRC
[17:27:20] *** Andrius has quit IRC
[17:27:37] *** Gottaname has joined #bittorrent
[17:38:19] *** Waldorf has quit IRC
[17:49:28] *** kwinz2 has joined #bittorrent
[17:52:14] *** _rafi_ has joined #bittorrent
[18:02:07] <choykloun> jesus tapdancing christ
[18:02:12] <choykloun> 1.8 million DHT nodes in my logs
[18:02:25] <TheSHAD0W> That's all?
[18:02:34] <choykloun> well its from a few hrs of running :p
[18:02:50] <choykloun> without purposedly trying to discover as many peers as possible :P
[18:04:58] <choykloun> well atleast the code works well under real-life circumstances
[18:05:42] <choykloun> needed to optimize some minor shit
[18:06:28] <choykloun> anyway, crypto crypto crypto !
[18:07:25] *** teamcoltra has quit IRC
[18:14:36] <The_8472> your implementation is inefficient
[18:14:40] <choykloun> ya
[18:14:45] <choykloun> its highly experimental
[18:15:15] <The_8472> 1d, 14h uptime and i've seen sent/receieved 300k packets. that includes my own queries and incoming ones
[18:15:20] <choykloun> its not something that would be used outside lab or limited applications
[18:15:36] <choykloun> ya
[18:15:55] <choykloun> its not general purpose
[18:15:59] <choykloun> not the current algorithm atleast
[18:16:13] <The_8472> *350k packets reply/response pairs. 290k of those are incoming ones.
[18:16:32] <The_8472> meaning i'm more efficient than the average :)
[18:17:28] * The_8472 pours molten tungsten into the channel
[18:17:43] <The_8472> http://forum.bittorrent.org/viewtopic.php?pid=889#p889 <-
[18:17:46] <choykloun> but mine is probably gonna get your torrent to more nodes :P
[18:18:10] <The_8472> get_peers lookups don't have to be and shouldn't be exhaustive
[18:18:25] <The_8472> i could make mine exhaustive, but that's highly inefficient
[18:18:31] <choykloun> some of my get_peers are from an old hack
[18:18:36] <choykloun> before i had real transaction management
[18:18:43] <choykloun> needed coz of token crap
[18:39:52] *** Andrius has joined #bittorrent
[18:44:44] *** bt42 has joined #BitTorrent
[18:51:59] *** bittwist has quit IRC
[19:12:36] *** kwinz2 has quit IRC
[19:32:31] *** HandheldPenguin` is now known as HandheldPenguin
[20:06:34] *** kwinz2 has joined #bittorrent
[20:11:44] *** HandheldPenguin is now known as HandheldPenguin`
[20:25:53] <choykloun> hey
[20:25:56] <choykloun> the_8472 or anyone alive ?
[20:27:12] *** Andrius has quit IRC
[20:27:19] <TheSHAD0W> I'm still alive...
[20:27:21] *** Andrius[] has joined #bittorrent
[20:27:48] <choykloun> ok
[20:28:17] <choykloun> i played around a bit and was able to easily bring down a web hosting company i hate by (ab)using DHT
[20:28:24] <TheSHAD0W> X_X
[20:28:50] <choykloun> so uhm, there is a DoS problem ...
[20:28:55] <choykloun> and i havent even optimized it
[20:29:01] <choykloun> or really violated the letter of the protocol spec even
[20:30:01] <TheSHAD0W> ...And your ECC addition would help solve the issue?
[20:30:36] <choykloun> does every technical thing i talk about have to be related just because i mention them on the same day? :)
[20:31:19] *** ryanprior has joined #bittorrent
[20:31:43] <TheSHAD0W> LOL
[20:31:45] <TheSHAD0W> Seriously.
[20:32:13] <TheSHAD0W> If there's an amplification problem in DHT, it needs a solution ASAP.
[20:32:19] <choykloun> ok
[20:32:24] <choykloun> let me just see HOW high it is
[20:32:34] <TheSHAD0W> Mole would suggest using hashcash...
[20:32:45] <choykloun> currently the host i included in the list of targets for monitoring is getting about 40 connections/sec to port 80
[20:32:50] <choykloun> ya
[20:32:56] <choykloun> thats what i have suggested from the start :p
[20:33:04] <choykloun> though i call it client-side puzzles :P
[20:34:04] <choykloun> anyway
[20:34:43] <choykloun> i wanna see how high i can get the rate
[20:35:14] <choykloun> and what, if anything, my dear hated test subject, will do about it :>
[20:35:39] <TheSHAD0W> ...
[20:35:43] <TheSHAD0W> Please don't.
[20:35:44] <TheSHAD0W> :-P
[20:36:07] <choykloun> if you'd ever met him you'd do the same thing
[20:36:10] <choykloun> and it's so perfect
[20:36:27] <choykloun> anything new (D)DoS related you have a small network with real traffic to test it on!
[20:37:22] <choykloun> and at most a few 14 year old kids pages go down
[20:37:24] <choykloun> win-win-win
[20:38:25] <TheSHAD0W> Unless you source enough traffic to cause upstream problems.
[20:38:29] <choykloun> nah
[20:38:33] <choykloun> this isnt much bw really
[20:38:40] <choykloun> its more a cpu intensive attack
[20:39:05] <choykloun> i mean i even attack oen of my own hosts just to get some idea of the rates
[20:42:16] <choykloun> and it actually provides good input on how to mitigate ddos attacks
[20:42:43] <choykloun> as i can study what services go down first at what rates etc..
[20:46:06] <choykloun> but... last i heard it was absolutely impossible to use dht for anything like this
[20:46:13] <choykloun> at most 3x udp amplification
[20:46:14] <choykloun> ...?
[20:46:15] <choykloun> :)
[20:51:44] <The_8472> well, the DHT is lossy, so the question is if we're actually talking about amplification here or just using the DHT as reflector to spread the IPs
[20:54:38] <choykloun> well the traffic continues for atleast 1h after i kill the dht client... i guess that could be called amplification, no? :)
[20:55:44] <The_8472> what exactly are you doing?
[20:55:53] <choykloun> will explain tomorrow!
[20:56:15] <choykloun> im too tired atm
[20:56:19] <The_8472> because 1h seems longer than most timeouts should be
[20:56:36] <choykloun> ive seen that doing perfectly legit dht stuff too
[20:57:14] <The_8472> sounds like lazy routing table implementations then
[20:58:00] <choykloun> attack itself isnt dependent on implementation
[20:58:07] <choykloun> but stuff like that certainly makes it worse haha
[20:58:09] <The_8472> well... thinking of it... local buckets can even retain stale nodes for a long time *scratching head*
[21:01:30] <choykloun> by the way, you are aware that i'm _extremely_ experienced in defending against ddos attacks? :>
[21:01:54] <The_8472> yes you have mentioned that, several times
[21:02:13] <choykloun> and yet any attack based on dht is impossible... :)
[21:04:21] <choykloun> good example of things not being that impossible: http://www.securityfocus.com/news/493
[21:05:34] <choykloun> and possibly a hint to why i simply cant stand jch and his 'alarmist symantec tinfoilhat blahblah' speech
[21:06:26] <DWKnight> [4:14:35pm] <choykloun> good example of things not being that impossible: http://www.securityfocus.com/news/493 <-- also more than a touch dated
[21:06:27] <The_8472> :roll:
[21:06:31] <choykloun> yeah
[21:06:36] <choykloun> but a damn funny story back then
[21:07:54] <The_8472> i'm just saying that not all attacks are worth defending against. e.g. when you get an amplification of 1.5 under labconditions i simply wouldn't care about that... it's like just getting a few more zombies into your botnet or a extra line
[21:08:00] <choykloun> and the funniest thing about it is that we made people believe we've had the bug for ages... while in reality i sat down, examined exactly what happened, and hurridly wrote an exploit :P
[21:08:20] <choykloun> the_8472: small amp factors are mostly a matter of good hygiene
[21:08:36] <choykloun> but dht also has that added unique factor that it freely hands over 1.5M addrs
[21:08:55] <The_8472> on the other hand defending against them decreases efficiency. so i'd be careful with that
[21:09:01] <The_8472> using the DHT as bouncer to spread out IPs on the other hand might be a valid concern.
[21:09:18] <choykloun> its possible to find the right balance(s)
[21:09:48] <The_8472> probably.
[21:11:02] <swolchok> choykloun: I've got scans of Vuze DHT that show almost 1 million  from 30 minutes of scanning
[21:11:08] <choykloun> ya
[21:11:14] <choykloun> exactly my point
[21:11:45] <The_8472> mainline DHT should be around 4M of reachable nodes atm, according to my estimator
[21:12:10] <The_8472> more if you include the non-reachable ones
[21:13:08] <swolchok> does mainline do the same thing as Vuze where it will propagate IPs in response to find_node requests even if they're not known to be alive?
[21:13:39] <swolchok> it's so much harder to answer those questions on mainline because the main impl isn't open source :(
[21:15:05] <TheSHAD0W> I think you need to distinguish mainline from utorrent.
[21:15:20] <TheSHAD0W> Even if utorrent is technically the main distro now.
[21:15:23] <swolchok> well, mainline itself is very underspecified.
[21:15:45] <swolchok> so it seems to me that the answers to questions like that depend on what most nodes actually do.
[21:19:00] <choykloun> DHT is much better designed and specified than many commercial protocols...
[21:19:37] <The_8472> <swolchok> well, mainline itself is very underspecified. <- i mostly go by the kademlia paper
[21:19:46] <kjetilho> what's commercial protocols?
[21:19:46] <choykloun> for totally unimportant stuff like burglary and fire alarm transmission...
[21:20:18] <kjetilho> yeah, proprietary protocols are generally designed by people with no experience
[21:20:18] <swolchok> The_8472: fair enough, if most everyone else does. the behavior of Vuze I was referring to is, IIRC, in contradiction to the Kademlia paper.
[21:20:39] <choykloun> you should see the Security Industry Association alarm transmission protocol
[21:21:04] <choykloun> the latest version of it starts with how it should be modulated in 300 baud over phone lines
[21:21:15] <choykloun> and includes stuff like analog audio transmission
[21:21:25] <choykloun> and this is what people cram into tcp/ip today
[21:21:54] <choykloun> also everyone does wrong in some way and/or has their own idea of what it 'should' look like
[21:22:31] <choykloun>   MODCALL(NUMmodcall_register_callback)(MODULE_NAME, MODULE_NAME".data_in", sia_data_in_callback);
[21:22:34] <choykloun>   MODCALL(NUMmodcall_register_callback)(MODULE_NAME, MODULE_NAME".comip_data_in", sia_comip_data_in_callback);
[21:22:38] <choykloun>   MODCALL(NUMmodcall_register_callback)(MODULE_NAME, MODULE_NAME".iris_data_in", sia_iris_data_in_callback);
[21:22:41] <choykloun>   MODCALL(NUMmodcall_register_callback)(MODULE_NAME, MODULE_NAME".text_data_in", sia_text_data_in_callback);
[21:23:00] <choykloun> havent found any more perversions of it ... yet
[21:24:11] <choykloun> we have wonderful stuff like emergency response centers not seeing the alarm type if its missing a certain parameter thats not even relevant
[21:25:27] <choykloun> and of course they can only receive one format, over one transport (rs232)
[21:49:01] *** kwinz2_ has joined #bittorrent
[22:06:24] *** kwinz2 has quit IRC
[22:18:07] *** rrr_ has quit IRC
[22:31:37] <The_8472> <swolchok> The_8472: fair enough, if most everyone else does. the behavior of Vuze I was referring to is, IIRC, in contradiction to the Kademlia paper. <- the kademlia paper does not dictate that you have to ascertain reachability before you hand out node addresses
[22:32:00] <The_8472> especially not if you're using the revised paper's optimized contact accounting, i.e. replacement buckets
[22:33:08] <choykloun> ok, real-life testing concluded:
[22:33:13] <choykloun> easily brings down a web server farm
[22:33:13] <The_8472> that's how i have implemented it too. nodes may get inserted into the routing table under some conditions even if they haven't been verified to be reachable. but that reachability will get verified at some point
[22:33:21] <choykloun> without even violating the specifications
[22:33:49] <swolchok> The_8472: that has bad ramifications for DDoS
[22:34:09] <swolchok> injected IPs stay in the table longer after injection even if they're not reachable
[22:34:11] <The_8472> it significantly improves efficiency though
[22:34:27] <The_8472> and it only applies to the local bucket in my case
[22:34:52] <The_8472> buckets closer to the root get cleaned out pretty fast
[22:36:15] <The_8472> a good DHT implementation retains the oldest entries in its buckets, thus new nodes shouldn't be able to get inserted into anything but the local buckets and those see maintenance every 10 minutes or so in my case
[22:37:15] <The_8472> the problem is... if you don't do it your closest-node sets may get inaccurate, and those are a cornerstone of DHT routing.
[22:38:02] <The_8472> <choykloun> easily brings down a web server farm <- try whitehouse.gov ^^
[22:38:08] *** goussx_ has joined #bittorrent
[22:38:25] <choykloun> nah, this guy was more fun
[22:38:42] <DWKnight> riaa?
[22:38:44] <choykloun> and it had the positive side effect of him promising to leave me the fuck alone and vice versa :)
[22:43:12] <kjetilho> so what's the kind of amplification you're achieving?
[22:43:31] <choykloun> will describe everything tomorrow!
[22:43:44] <choykloun> xmas party, work, coding, 5am, etc does not make for proper verbality
[22:45:51] <kjetilho> the sad fact is that you don't need much bandwidth to take down a normal webserver.  if you have a gigabit, you'll do fine.
[22:46:02] <choykloun> its not a bw attack
[22:46:07] <choykloun> its a connection flood
[22:46:21] <kjetilho> right.
[22:46:23] <choykloun> with no upper limit on rate or number of sources
[22:46:30] <kjetilho> but you need bandwidth to send packets :)
[22:46:40] <choykloun> not much :)
[22:46:53] <kjetilho> that's what I was saying
[22:46:56] <choykloun> anyways this was just a quick hack
[22:47:12] <The_8472> connection flood? those are UDP packets. there are no connections unless you do stupid things like conntrack
[22:47:15] <choykloun> and it also had the positive side effect of me and my unwilling test subject agreeing to stay the fuck away from each other in the future :)
[22:47:40] <choykloun> 21:57:53.903449 IP 173.31.33.203.63738 > 212.117.163.41.8443: S 4042217963:4042217963(0) win 8192 <mss 1460,nop,nop,sackOK>
[22:47:44] <choykloun> 21:57:53.905707 IP 93.167.81.167.60897 > 212.117.163.41.8443: S 1781138788:1781138788(0) win 8192 <mss 1460,nop,wscale 2,sackOK,timestamp 12098763 0>
[22:47:47] <choykloun> 21:57:53.906188 IP 114.38.70.79.3115 > 212.117.163.41.8443: S 1884057994:1884057994(0) win 65535 <mss 1440,nop,nop,sackOK>
[22:47:51] <choykloun> 21:57:53.927893 IP 94.197.201.73.54615 > 212.117.163.41.8443: S 1186152206:1186152206(0) win 8192 <mss 1400,nop,nop,sackOK>
[22:47:54] <choykloun> 21:57:53.929579 IP 114.42.183.21.50739 > 212.117.163.41.8443: S 235247674:235247674(0) win 8192 <mss 1440,nop,nop,sackOK>
[22:47:58] <choykloun> 21:57:53.956489 IP 218.173.5.1.1835 > 212.117.163.41.8443: S 3828581527:3828581527(0) win 65535 <mss 1440,nop,nop,sackOK>
[22:47:58] <kjetilho> choykloun: please learn to use a pastebin!
[22:48:00] <The_8472> stop spamming ffs...
[22:48:01] <choykloun> 21:57:53.957527 IP 62.135.89.46.52945 > 212.117.163.41.8443: S 1639047678:1639047678(0) win 8192 <mss 1452,nop,nop,sackOK>
[22:48:05] <choykloun> 21:57:53.963980 IP 113.254.181.186.1466 > 212.117.163.41.8443: S 1077231375:1077231375(0) win 65535 <mss 1460,nop,nop,sackOK>
[22:48:08] <choykloun> 21:57:53.986877 IP 221.127.46.250.60461 > 212.117.163.41.8443: S 1244939316:1244939316(0) win 65535 <mss 1452,nop,nop,sackOK>
[22:48:12] <choykloun> 21:57:53.996662 IP 151.57.47.86.52610 > 212.117.163.41.8443: S 14177893:14177893(0) win 8192 <mss 1360,nop,nop,sackOK>
[22:48:15] <choykloun> udp packets? :P
[22:48:19] <choykloun> it didnt interrupt any conversation this time EITHER!
[22:48:24] <The_8472> ... we don't care
[22:48:34] <The_8472> it scrolls previous conversation away == disruptive
[22:48:35] <choykloun> ok!
[22:48:36] <choykloun> damn
[22:48:40] <kjetilho> choykloun: because we waited for it to stop flooding our windows
[22:48:43] <choykloun> you do everything the oposite way :)
[22:48:59] <kjetilho> no, using a pastebin is standard procedure in *every* channel I'm in
[22:49:04] <The_8472> same
[22:49:22] <choykloun> not when nobody's interrupted
[22:49:26] <choykloun> but ok ok!
[22:49:26] <The_8472> even then
[22:49:30] <kjetilho> same
[22:50:16] <The_8472> anyway, if it's TCP connections that would mean you're announcing things
[22:50:24] <choykloun> its a bit complicated
[22:50:26] <The_8472> and those shouldn't be spoofable easily
[22:50:27] <choykloun> :p
[22:50:35] <choykloun> its not entirely straightforward to explain all the finer points
[22:50:42] <choykloun> but basically im modifying get_peers results
[22:51:05] <The_8472> hrrm... interesting
[22:51:10] <choykloun> and using certain strategies to make sure i get enough queries to achieve a good attack
[22:51:19] <choykloun> this is the quick-hack version
[22:51:26] <choykloun> can do it on a network-wide scale
[22:51:32] <The_8472> i.e. inserting yoruself close to popular keys
[22:52:00] <The_8472> i can already think of a few defenses, some of which i've already implemented partially ^^
[22:52:15] <choykloun> there are some other fine details
[22:54:23] *** goussx has quit IRC
[22:54:23] *** goussx_ is now known as goussx
[22:54:35] <The_8472> though i'd like to avoid filtering by /24s... since that'll become useless once you move to v6 anyway.
[22:54:47] <choykloun> this was done using a single node id
[22:55:01] <The_8472> ah well, then it should be easy to defend against
[22:55:03] <choykloun> well i switched a few times for experimental purposes
[22:56:27] <choykloun> the REAL version is doing it with a bunch of hacked computers and nodes spread out all over the network :)
[22:57:43] <choykloun> but yeah lets walk this through tomorrow
[22:57:50] <swolchok> since he seems to have a DHT DDoS anyway, my version is to get some IDs close to your target's ID (a non-step in mainline, easy enough in Vuze) and insert the target into the routing tables
[22:58:18] <The_8472> well, if you do what i think you're doing then all that's needed is filtering result sets from get_peer lookups to eleminate duplicate IPs (not to mention ports, which already should be the case)
[22:58:22] <swolchok> the trick is you need to do the insert to the neighborhood of the target
[22:58:35] <The_8472> |23:01:46| <The_8472> i.e. inserting yoruself close to popular keys <-
[22:58:36] <The_8472> ;)
[22:58:53] <choykloun> can do this just fine without duplicates
[22:59:54] <choykloun> btw my box is still receiving about 40 conn/sec
[23:00:07] <The_8472> mhhh... well, write up an explanation and post it on the forum
[23:00:41] <choykloun> write up yes; forum no, will make it available to the proper guys here
[23:02:10] <The_8472> :roll:
[23:02:15] <The_8472> stop acting like a small child
[23:02:41] <The_8472> if you want something fixed then post it somewhere non-ephemeral
[23:02:57] <The_8472> didn't ask you to actually read what anyone responds to your post
[23:03:00] <The_8472> just post it
[23:03:19] <swolchok> I bet he's just sybiling to get enough queries
[23:03:37] <swolchok> and apparently injecting the target into get_peers responses
[23:03:56] <The_8472> we shall see
[23:05:16] <The_8472> mainline DHT estimator says 3.9M nodes
[23:06:12] <swolchok> he can sybil on at LEAST 2^16 ports using completely distinct 16-bit prefixes
[23:06:15] <choykloun> there are some finer touches to how it's done but its not difficult in itself
[23:06:16] <swolchok> **ID prefixes
[23:06:30] <choykloun> and im using ONE node id, host and port
[23:06:40] <choykloun> except when changing for experimental purposes
[23:08:10] <swolchok> well, if nodes are idiotic, just ping a bunch of them to get set up
[23:08:15] <swolchok> (DHT ping)
[23:08:22] <choykloun> since you're already aware of how weird i can be you can probably figure out that i implement stuff in weird ways :)
[23:08:49] <swolchok> nah, I'm just an academic so I think of everything in terms of my previous work.
[23:09:02] <choykloun> ah
[23:09:09] <choykloun> im a high-school dropout :)
[23:09:29] *** kwinz2_ has quit IRC
[23:09:31] <The_8472> swolchok, getting routing table management right is hard, i meticulously tinkered on our mldht imeplementation for days. And i still find things that can be tweaked on occasion
[23:10:29] <choykloun> ya
[23:10:42] <choykloun> often behavior becomes so complex you could as well be trying to predict the weather..
[23:12:08] <choykloun> its damn fun though :)
[23:12:09] <The_8472> well, yes. you have to understand emergent behavior of it. like some maintenance task in the task scheduler here, combined with a last-incoming counter there and suddenly your routing table survives connection dropouts
[23:13:16] <choykloun> well it usually ends up more like something resembling chaos theory :)
[23:13:32] <The_8472> nah, it's not that bad
[23:13:32] <choykloun> especially when you have several different implementations etc
[23:13:46] <The_8472> things are still well-structured
[23:13:58] <choykloun> ya dht is very nicely done
[23:13:58] <The_8472> it's not a small world net, it's a structured overlay
[23:14:15] <choykloun> but its still one of those self-organizing systems
[23:14:35] <choykloun> where you cant strictly predict the outcome of your actions
[23:15:40] <The_8472> sure, but you can consider the expected, average impact
[23:15:44] <choykloun> which is part of the charm
[23:15:55] <choykloun> same thing applies to bgp for example
[23:16:01] <choykloun> or even power grid design
[23:17:18] <The_8472> oldest node in my routing table: 101k sec
[23:17:39] <The_8472> 28h
[23:19:24] <The_8472> ah, found an even older one in the v6 table. 356k sec
[23:19:53] <The_8472> 4d
[23:33:30] <choykloun> still getting connections .. :)
[23:38:38] <The_8472> well, that's more part of the BT clients keeping a cache of connection canidates than part of the DHT
[23:39:04] <choykloun> interesting in any case
[23:40:03] <choykloun> anyways
[23:40:12] <choykloun> far too tired for anything technical right now
[23:46:25] *** _rafi_ has quit IRC
[23:48:32] <choykloun> i spend like 6h with http://anakata.prq.to/temp/os360ipl.png today, so .. :)
[23:56:43] *** n215_ has joined #bittorrent





top