[00:01:49] *** kad has joined #gerrit[00:34:28] *** qchris_away is now known as qchris[00:35:27] *** dougsland has joined #gerrit[00:53:03] *** signed8bit has joined #gerrit[00:59:51] *** pcambra has joined #gerrit[01:23:12] *** pcambra has quit IRC[01:26:45] *** qchris has quit IRC[01:35:31] *** signed8bit is now known as signed8bit_ZZZzz[01:39:03] *** signed8bit_ZZZzz is now known as signed8bit[01:56:14] *** dougsland has quit IRC[02:27:27] *** signed8bit has quit IRC[02:43:07] *** teran has quit IRC[02:48:37] *** teran has joined #gerrit[03:06:17] *** pcambra has joined #gerrit[03:10:53] *** pcambra has quit IRC[03:26:19] *** Vampire0 has joined #gerrit[03:26:30] *** Vampire0 has quit IRC[03:26:30] *** Vampire0 has joined #gerrit[03:29:40] *** Vampire0_ has quit IRC[03:52:17] *** p4plus2 has quit IRC[04:34:52] *** spudly has quit IRC[04:35:24] *** spudly has joined #gerrit[04:54:25] *** pcambra has joined #gerrit[04:59:57] *** pcambra has quit IRC[05:58:28] *** talios has quit IRC[06:27:31] *** gerritUser has quit IRC[06:38:06] *** miqui has joined #gerrit[06:42:44] *** pcambra has joined #gerrit[06:48:30] *** pcambra has quit IRC[07:01:57] *** talios has joined #gerrit[07:22:22] *** [HeOS] has quit IRC[08:30:58] *** pcambra has joined #gerrit[08:45:57] *** pcambra has quit IRC[08:56:55] *** HeOS has joined #gerrit[09:03:16] *** srenatus has joined #gerrit[09:11:38] *** miqui has quit IRC[09:17:13] *** mountains__ has joined #gerrit[09:26:48] *** Camel has quit IRC[09:26:51] *** drag00n has quit IRC[09:27:16] *** Camel_ has joined #gerrit[09:27:19] *** teran has quit IRC[09:27:24] *** drag00n has joined #gerrit[09:34:33] *** St_Marx has quit IRC[09:38:43] *** franred has joined #gerrit[10:11:59] *** [HeOS] has joined #gerrit[10:12:12] *** talios has quit IRC[10:14:58] *** HeOS has quit IRC[10:17:32] *** teran has joined #gerrit[10:18:28] *** teran_ has joined #gerrit[10:22:15] *** teran has quit IRC[10:29:56] *** pcambra has joined #gerrit[10:32:53] *** timothy has joined #gerrit[10:34:53] *** pcambra has quit IRC[12:05:20] *** dougsland has joined #gerrit[12:07:43] *** qchris has joined #gerrit[12:08:17] *** [HeOS] is now known as HeOS[12:13:04] *** teran_ has quit IRC[12:18:03] *** pcambra has joined #gerrit[12:22:01] *** arbmacp has joined #gerrit[12:22:45] *** pcambra has quit IRC[12:29:09] *** arbmacp has quit IRC[13:02:23] *** plinio has joined #gerrit[13:19:37] *** teran has joined #gerrit[13:20:18] *** teran_ has joined #gerrit[13:24:02] *** teran has quit IRC[14:09:27] *** St_Marx has joined #gerrit[14:27:11] *** teran has joined #gerrit[14:30:09] *** teran__ has joined #gerrit[14:30:52] *** teran_ has quit IRC[14:33:17] *** teran has quit IRC[14:48:46] *** hugares has joined #gerrit[15:40:21] *** St_Marx has quit IRC[15:44:37] *** St_Marx has joined #gerrit[15:58:31] *** teran__ has quit IRC[15:59:01] *** qchris is now known as qchris_meeting[15:59:44] *** teran has joined #gerrit[16:00:22] *** teran_ has joined #gerrit[16:04:09] *** teran has quit IRC[16:21:46] *** qchris_meeting is now known as qchris_away[16:23:36] *** Carmela has quit IRC[16:26:23] *** Carmela has joined #gerrit[16:29:46] *** qchris_away is now known as qchris_meeting[17:01:24] *** mountains__ has quit IRC[17:10:21] *** tobiasvl has quit IRC[17:10:40] *** tvl has joined #gerrit[17:12:18] *** tvl is now known as tobiasvl[17:17:42] *** qchris_meeting is now known as qchris_away[17:31:03] *** qchris_away is now known as qchris_meeting[17:47:56] *** devlaps has joined #gerrit[18:34:50] *** qchris_meeting is now known as qchris[18:36:38] *** pedahzur has joined #gerrit[18:39:06] <pedahzur> Hello again...I'm back. Here is the thread dump I'm seeing a lot when a user is trying to push a big update to the binary blob. https://gist.github.com/jkugler/a201e607d3fa466937e0 I'm not Java expert, but parking to wait sounds like a slow operation to me.[18:42:01] <pedahzur> dougk, and any other developer that would like to chime in. ^^[18:43:35] *** qchris is now known as qchris_away[18:43:46] *** teran_ has quit IRC[18:48:55] *** teran has joined #gerrit[18:57:29] *** Vampire0 has quit IRC[18:59:51] *** teran has quit IRC[19:00:20] *** teran has joined #gerrit[19:01:46] *** qchris_away is now known as qchris_meeting[19:04:46] *** timothy has quit IRC[19:32:22] *** hugares has quit IRC[19:33:37] *** hugares has joined #gerrit[19:49:46] *** qchris_meeting has quit IRC[20:04:50] <dougk> pedahzur: my experience that that's not been an issue (but maybe if it's being blocked by something improperly, it could explain the issue?)[20:08:22] <pedahzur> dougk: Yeah, I don't know...huge pipe on both ends, no I/O or CPU load on the server...45K/sec to 85K/sec uploads. We're at a loss to explain. Anybody offer commercial support for Gerrit? :)[20:09:06] <dougk> and you're not using Windows on the client side, right? :P[20:11:53] <dougk> as for commercial support, I believe Luca (GerritForge) may offer services.[20:13:11] *** srenatus has quit IRC[20:13:34] <pedahzur> dougk: Yeah, pretty sure it's a Mac client.[20:16:14] <dougk> the only time I can think of that we've seen abysmally slow transfer is because Windows versions of the client have an absolutely sickeningly old version of openssh and openssl[20:16:29] <dougk> this got fixed with the latest Git for Windows, though.[20:17:11] <dougk> I mean, if you've repacked the repo, done garbage collection, etc. and still see the slowness... it may be a good time to ask on the mailing list, too.[20:30:30] *** hugares has quit IRC[20:51:02] *** plinio has quit IRC[21:01:57] *** teran has quit IRC[21:25:15] *** russt has joined #gerrit[21:35:54] *** talios has joined #gerrit[21:55:47] *** devlaps has quit IRC[21:57:17] <pedahzur> dougk: Haven't repacked the repo. Need to try that. Right now, I have two Jenkins jobs pulling from the repo with the large binaries. Both are showing "java.lang.Thread.State: WAITING (on object monitor)" and it's bringing the rest of the system to a crawl.[21:58:36] <pedahzur> dougk: I'm still not clear on how a repack will help. The git-repack man page says "This command is used to combine all objects that do not currently reside in a "pack", into a pack." That sounds like it will create more compression/expansion overhead, not reduce it.[22:00:43] <dougk> poorly-packed repositories tend to increase overhead; Gerrit actually repacks repos on the fly to each client, as I understand it...[22:07:42] *** devlaps has joined #gerrit[22:10:59] *** teran has joined #gerrit[22:13:54] <pedahzur> dougk: Hence, "SSH git-upload-pack '/repo-name'" for clone operation. But why doesn't that use CPU or disk? Gerrit will be near unusable with these jobs running, but CPU and DIsk I/O is at near zero.[22:14:34] <pedahzur> dougk: I assume a git repack is safe to run while other processes are accessing the repo?[22:21:52] <dougk> yes, repack (like gc) are atomic; nothing actually changes until the point at which it's finished, as I recall.[22:22:34] <dougk> I think some temporary files (or even the pack as it's being constructed) are written, but the original objects aren't removed until the pack is done.[22:36:33] <jkt> pedahzur: do you have some contention around DB access?[22:38:55] <pedahzur> jkt: No, plenty of connections. No load on the DB server. Also, it wouldn't be doing lots of DB updates for a repo during a clone, woulod it?[22:40:13] <jkt> "plenty of connections" was what I was referring to; make sure you have enough connection slots on both sides, i.e. allowed within the DB and also enabled within Gerrit[22:41:39] <pedahzur> jkt: The gerrit crawl only happens (it seems) when these repos-with-large-binaries are being accessed. Right now, we have two "SSH git-upload-pack '/chroma-externals' processes in the thread dumps waiting with state java.lang.Thread.State: WAITING (on object monitor)[22:42:50] <jkt> cannot help, sorry[22:43:23] <pedahzur> jkt: Any idea what "object monitor" is and why it's waiting on it? :)[22:43:54] <jkt> nope[23:15:00] *** misprint has quit IRC