[00:02:07] *** JasonO has quit IRC [00:06:51] *** JasonO has joined #chromium-os [00:06:51] *** JasonO has joined #chromium-os [00:07:36] *** magn3ts has quit IRC [00:14:55] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "x86-alex-binary" from df9ed46aa47c92c432edcbbac6e19ccd69753020: Gaurav Shah <gauravsh at chromium dot org>, mirandac at chromium dot org <mirandac at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>)' [00:15:57] *** sadrul has quit IRC [00:16:19] <gauravsh> not me [00:17:22] <cmasone> chrome sig 11 [00:17:24] <cmasone> 17899 [00:17:43] *** truty has quit IRC [00:20:11] *** magn3ts has joined #chromium-os [00:21:32] *** behdad has quit IRC [00:27:37] <pstew> chromeos-cryptohome-0.0.1-r84: mount_unittest.cc:678: Failure [00:27:39] <pstew> chromeos-cryptohome-0.0.1-r84: Expected: (100000) > (last_activity_delay_us_2), actual: 100000 vs 137459 [00:27:41] <pstew> chromeos-cryptohome-0.0.1-r84: [ FAILED ] MountTest.UserActivityTimestampUpdated (385 ms) [00:27:53] <cmasone> pstew: recent change in cryptohome? [00:28:08] <cmasone> crosbot: announce +tree [00:28:08] <crosbot> cmasone: Admin access required [00:28:15] <cmasone> crosbot: stat tree [00:28:15] <crosbot> cmasone: tree: Tree is closed (Automatic: "cbuildbot" on "x86-alex-binary" from df9ed46aa47c92c432edcbbac6e19ccd69753020: Gaurav Shah <gauravsh at chromium dot org>, mirandac at chromium dot org <mirandac at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>) [00:28:48] <ellyjones> cmasone: it's always been announcing tree [00:28:52] <ellyjones> hasn't it? [00:28:55] <ellyjones> crosbot: announce +tree [00:28:56] <crosbot> ellyjones: announcing tree [00:28:58] <ellyjones> yeah [00:29:02] <pstew> I've been looking for a change to cryptohome but nothing in the blamelist seems to have changed that. [00:29:04] <ellyjones> crosbot: admin +cmasone [00:29:05] <crosbot> ellyjones: ellyjones cmasone [00:29:12] <pstew> It's also not off by all that much (30%) [00:29:39] <gauravsh> the last change to cryptohome was 8 days ago, so it is unlikely it's that [00:29:47] <pstew> I can assign to the chrome crash bug and see if this persists afterwards. [00:29:54] <cmasone> wait, what bot is this? [00:30:11] <cmasone> because the pfq just blew up too, and that's a powerd sigabrt [00:30:47] <cmasone> pstew: chrome crash?? This is a cryptohome unit test failure...why would we assign this to the chrome crash bug?? [00:30:58] <pstew> chromeosbuild9 [00:31:04] *** wfrichar has quit IRC [00:31:19] <cmasone> pstew: what builder? [00:31:19] <pstew> cmasone: Chrome crashed in this same log as well. [00:31:38] <cmasone> pstew: people are already trying to get us to the point where we can debug that [00:31:51] <cmasone> pstew: if this unit test went flaky, that's a new issue [00:31:52] <cmasone> also [00:31:56] <cmasone> there's this powerd thing [00:32:20] <pstew> buildername: x86-alex-binary, but you should know that, so I'm guessing you want the answer to something else...? [00:32:43] <gauravsh> I wonder if the flaky unittest thing is related to the system clock being adjusted midway through the test [00:32:48] <cmasone> pstew: I did not know that, because there are multiple red bots; that was just the first one [00:33:19] <cmasone> pstew: and there have been no tree status updates since the tree went red, so I don't know what's being investigated and by whom :-) [00:34:59] <pstew> Okay. I'll update the tree status to say I suspect cryptohome flakiness (nothing has changed in cryptohome recently as per gaurav + my inspection of the blame list). The log also shows chrome crashing and a change to localtime during the test run. [00:35:20] <cmasone> pstew: what about this powerd crash during every test on the pfq? [00:36:37] <pstew> I haven't seen this. I guess there's another parallel issue? [00:36:54] <pstew> I have a dental appt coming up which I need to leave for in 5. [00:37:10] <cmasone> pstew: do you have someone covering? Puneet left too, right? [00:37:30] <cmasone> pstew: and yes...there's another parallel issue, it's the biiiiiig red bot on the waterfall :-) [00:37:41] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "x86-alex-binary" from df9ed46aa47c92c432edcbbac6e19ccd69753020: Gaurav Shah <gauravsh at chromium dot org>, mirandac at chromium dot org <mirandac at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>)' [00:37:41] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "x86-alex-binary" from df9ed46aa47c92c432edcbbac6e19ccd69753020: Gaurav Shah <gauravsh at chromium dot org>, mirandac at chromium dot org <mirandac at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>)' [00:41:35] *** JasonO has quit IRC [00:42:17] <crosbot> tree became 'Tree is open (chrome crashed again, sque applied power patches out of order, monitoring flake in cryptohome test)' [00:42:17] <crosbot> tree became 'Tree is open (chrome crashed again, sque applied power patches out of order, monitoring flake in cryptohome test)' [00:42:32] <cmasone> ellyjones: ooh...is it doubly announcing the tree now? [00:44:33] <pstew> Is anyone available for covering the tree for an hr while I have my dental appt (and Puneet is away as well)? [00:44:49] *** JasonO has joined #chromium-os [00:44:49] *** JasonO has joined #chromium-os [00:52:37] *** aaronp_ has quit IRC [00:52:58] *** tbarzic has quit IRC [00:57:29] *** tbarzic has joined #chromium-os [01:01:11] *** stevenjb has joined #chromium-os [01:01:11] *** ChanServ sets mode: +v stevenjb [01:05:15] <quiche> for the record, i'm covering as sheriff, for pstew [01:10:10] *** Kernel-Panic has joined #chromium-os [01:11:06] *** Kernel-Panic has joined #chromium-os [01:19:07] *** pstew_mobile_ has joined #chromium-os [01:28:58] *** magn3ts has quit IRC [01:30:22] *** pstew_mobile_ has quit IRC [01:30:28] *** Kernel-Panic has quit IRC [01:32:39] *** sadrul has joined #chromium-os [01:32:40] *** ChanServ sets mode: +v sadrul [01:38:10] *** JasonO has quit IRC [01:38:22] *** rsleevi is now known as rslllleevi [01:41:58] *** JasonO has joined #chromium-os [01:42:25] *** rslllleevi is now known as rsleevi [01:42:43] *** lipsinV2 has joined #chromium-os [01:44:24] *** behdad has joined #chromium-os [01:56:23] *** behdad has quit IRC [01:58:22] *** powderluv has quit IRC [02:28:41] *** petermayo has quit IRC [02:33:24] * quiche is gone. channel is sheriff. [02:33:51] *** mtennant has joined #chromium-os [02:33:51] *** ChanServ sets mode: +v mtennant [02:35:34] *** powderluv has joined #chromium-os [02:35:34] *** ChanServ sets mode: +v powderluv [02:38:50] *** jrbarnette has quit IRC [02:49:05] *** powderluv has quit IRC [02:58:23] *** superm1 has quit IRC [03:02:12] *** thieule has quit IRC [03:05:40] *** superm1 has joined #chromium-os [03:05:40] *** superm1 has joined #chromium-os [03:28:34] *** sergiu has quit IRC [03:42:46] <crosbot> tree became 'Tree is open (chrome crashed again, sque applied power patches out of order, monitoring flake in cryptohome test)' [03:42:46] <crosbot> tree became 'Tree is open (chrome crashed again, sque applied power patches out of order, monitoring flake in cryptohome test)' [03:49:56] *** stevenjb has quit IRC [03:55:20] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "tegra2_arthur-binary" from f930edd6264e33bc7a57ecc8cad52a80330cfb5e: _third_party_ at chromium dot org, cpu at chromium dot org <cpu at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>, derat at chromium dot org <derat at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>, dhollowa at chromium dot org <dhollowa at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>, koz at chromium dot org <koz at chromium dot org@0039d316-1c4b-4281-b [03:55:20] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "tegra2_arthur-binary" from f930edd6264e33bc7a57ecc8cad52a80330cfb5e: _third_party_ at chromium dot org, cpu at chromium dot org <cpu at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>, derat at chromium dot org <derat at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>, dhollowa at chromium dot org <dhollowa at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>, koz at chromium dot org <koz at chromium dot org@0039d316-1c4b-4281-b [04:08:21] <cmasone> crosbot: announce -tree [04:08:21] <crosbot> cmasone: announcing <none> [04:08:27] <cmasone> crosbot: announce +tree [04:08:27] <crosbot> cmasone: announcing tree [04:09:58] <cmasone> gauravsh: ping [04:09:58] *** JasonO has quit IRC [04:11:56] *** e_t_ has joined #chromium-os [04:12:32] *** magn3ts has joined #chromium-os [04:13:43] <crosbot> tree became 'Tree is closed (revision refs/heads/master in chromium/src/third_party/opencryptoki not found => gauravsh?)' [04:22:05] <gauravsh> cmasone: pong [04:22:08] <gauravsh> it can't be my change [04:22:18] <gauravsh> that went in 5 hours ago [04:22:28] <cmasone> it looks like those builders didn't pick it up [04:22:32] <gauravsh> it seems to not be picking up the latest manifest [04:22:36] <cmasone> and have the one iwth chromium not chromiumos [04:23:14] <gauravsh> yes, but repo should update the manifest before starting the build [04:23:23] <cmasone> I don't know what to do about it...I'd escalate to a trooper, but I don't see any :-) [04:23:46] *** petermayo has joined #chromium-os [04:23:46] *** ChanServ sets mode: +v petermayo [04:24:36] <gauravsh> i don't know what I can do about it - since I had reverted my original change (with the problem above) :) [04:25:04] <gauravsh> i just saw the ARM builds are also failing because of problems with initramfs [04:25:05] <cmasone> Maybe we force a new build? [04:25:10] <gauravsh> i concur [04:26:48] <cmasone> dun [04:26:59] <cmasone> arthur looks like it's on a bot with something else [04:27:18] <gauravsh> hmm, zgb-binary has been failing all day [04:27:25] <gauravsh> why was the tree not closing automatically then? [04:27:34] <cmasone> because continual failing != new tree closures [04:27:45] <cmasone> puneet said something about that bot earlier, there may be aknown issue [04:28:23] *** McMAGIC--Copy has quit IRC [04:28:30] <petermayo> all three of kaen, aebl and arthur appear to be on one bot. [04:28:36] <cmasone> oy [04:28:40] <petermayo> ve [04:28:41] <cmasone> ok, so that didn't work. [04:28:43] <cmasone> clobber? [04:28:45] <gauravsh> damn it [04:28:48] <gauravsh> yeah [04:29:22] <gauravsh> here i was getting down for a nice dinner, some wine, and a movie [04:29:59] <cmasone> gauravsh: ha! [04:30:21] <cmasone> Well, I have to get on a bus. WIll update status, so TPE can pick up [04:31:33] <crosbot> tree became 'Tree is closed (Manifest that was fixed hours ago not getting picked up by binary bots...clobbering)' [04:35:26] <gauravsh> cmasone: that seems to have done the trick. I see the sync stage succeeded this time. [04:35:32] <cmasone> gauravsh: yeah [04:35:38] <cmasone> didn't need to send that mail :-) [04:36:39] <gauravsh> hooray, me out [04:36:48] <gauravsh> please do not hesitate to not bug me :) [04:38:19] *** McMAGIC--Copy has joined #chromium-os [04:48:18] *** srao has quit IRC [05:17:26] *** oc80z has quit IRC [05:17:26] *** oc80z has joined #chromium-os [05:18:47] *** oc80z has joined #chromium-os [05:44:53] *** fluffman has quit IRC [05:58:03] *** behdad has joined #chromium-os [06:00:58] <crosbot> tree became 'Tree is closed (binary bots good) (need investigation of au test failures)' [06:05:24] <petermayo> Is anyone online that can take a look at what is causing relatively widespread au test failures? [06:07:06] <petermayo> Sorry - G'Night red tree. [06:07:11] *** petermayo has quit IRC [06:17:41] *** rsleevi has quit IRC [06:21:34] *** vmil86 has joined #chromium-os [07:05:41] *** behdad has quit IRC [07:12:53] *** behdad has joined #chromium-os [07:22:28] *** behdad has quit IRC [07:35:23] *** Adys has quit IRC [07:41:52] *** Adys has joined #chromium-os [07:43:56] *** Inumedia_ has quit IRC [07:45:05] *** Inumedia_ has joined #chromium-os [07:49:07] *** sakoman_ has quit IRC [07:49:29] *** sakoman_ has joined #chromium-os [08:08:01] *** marcheu has quit IRC [08:09:54] *** marcheu has joined #chromium-os [08:18:11] *** Inumedia_ is now known as Inumedia [08:18:21] *** Inumedia has joined #chromium-os [08:21:27] *** tbarzic__ has joined #chromium-os [08:40:50] *** e_t_ has quit IRC [08:43:27] *** tbarzic__ has quit IRC [08:49:22] *** Styx__ has joined #chromium-os [08:51:32] *** powderluv has joined #chromium-os [08:51:32] *** ChanServ sets mode: +v powderluv [08:59:49] *** Styx__ is now known as Styx [09:19:04] *** quannnum has quit IRC [09:19:58] *** quannnum has joined #chromium-os [09:23:14] *** sadrul has quit IRC [09:42:52] *** jochen__ has quit IRC [09:42:57] *** jochen__ has joined #chromium-os [09:42:58] *** ChanServ sets mode: +v jochen__ [09:49:25] *** sadrul has joined #chromium-os [09:51:38] *** Inumedia has quit IRC [09:52:11] *** Inumedia has joined #chromium-os [10:05:49] *** yusukes has quit IRC [10:08:02] *** Inumedia_ has joined #chromium-os [10:10:55] *** Inumedia has quit IRC [10:23:10] *** powderluv has quit IRC [10:31:36] *** BladeFreak has quit IRC [10:39:49] *** patcito has quit IRC [10:58:01] *** McMAGIC--Copy has quit IRC [11:33:18] *** McMAGIC--Copy has joined #chromium-os [11:36:00] *** Styx has quit IRC [11:46:05] *** vmil86 has quit IRC [12:06:56] *** Styx has joined #chromium-os [13:43:33] *** vmil86 has joined #chromium-os [14:14:38] *** Styx has quit IRC [14:25:39] *** rbyers has quit IRC [14:45:40] *** Inumedia_ has quit IRC [14:45:47] *** rharrison_chrome has joined #chromium-os [14:45:47] *** ChanServ sets mode: +v rharrison_chrome [14:56:58] *** sadrul has quit IRC [15:10:14] *** rbyers has joined #chromium-os [15:47:22] *** rharrison_chrome has quit IRC [15:50:13] *** Styx has joined #chromium-os [15:50:27] <njw> sheriff here. coffee, then looking at the latest and greatest on the tree. [15:54:30] *** Soak has joined #chromium-os [15:55:48] <Soak> hello, I have compiled and built chromium OS on my USB key but impossible to boot on (but I have selected Revomable device in the bios and removed other devices).. that say to me: insert your device and press a key [15:56:49] *** SoulRaven has joined #chromium-os [16:09:57] <Soak> no idea? :( [16:11:31] *** sadrul has joined #chromium-os [16:11:31] *** ChanServ sets mode: +v sadrul [16:11:45] <Soak> I think the problem is the file format [16:12:28] <Soak> I need FAT surely [16:14:52] *** rharrison_chrome has joined #chromium-os [16:14:53] *** ChanServ sets mode: +v rharrison_chrome [16:22:15] <ellyjones> Soak: you built an image, then dd'd it onto your stick? [16:23:56] *** lipsinV2 has quit IRC [16:25:47] <kliegs> looking into tree closure now [16:26:27] <kliegs> njw: ping - did you take a peak yet? [16:26:28] <njw> some of them are the segfaults mentioned in crosbug.com/17899 [16:26:44] <njw> haven't figured out anything else yet, and that one tidbit isn't too helpful. [16:27:57] <kliegs> it looks like x86 pineview full will likely go green soon - says eta 20 minutes and last build looks like repo flakiness [16:28:50] <kliegs> tegra2 full looks unhappy - errors in prebuilt.py [16:28:59] <kliegs> you were looking at the internal bots I take it then? [16:29:33] <njw> yeah. [16:30:09] <njw> I've got to run and do an interview - I'll cross my fingers for the x86 pineview just being a glitch. [16:30:26] <kliegs> x86 pineview full looks like repo flakiness. runs before were all good and its past that point [16:30:41] <kliegs> i'll go poke at prebuilt.py then move to internal bots [16:31:21] <Soak> ellyjones: I follow this: http://www.chromium.org/chromium-os/developer-guide [16:31:52] <Soak> so I have followed the tutorial, and do the ./image_to_usb.sh [16:32:00] <Soak> so the image is on my stick [16:32:03] <ellyjones> ok [16:32:08] <ellyjones> and then you plug it into what? [16:32:15] <Soak> but I think my computer don't recognize the usb stick [16:32:21] <Soak> like a boot device [16:32:38] <Soak> I plug into my PC [16:32:53] <Soak> and I modify the bios, to put only boot from a "removal device" [16:33:03] <Soak> and the PC don't want to boot on [16:33:57] <Soak> so I will try with FAT [16:34:45] <kliegs> sosa: I know you're not likely awake but ping for when you get in [16:36:51] <kliegs> For prebuilts - are those uploaded by the full builders or by the binary builders? [16:40:22] <crosbot> tree became 'Tree is closed (binary bots good) (need investigation of au test failures) (crosbug.com/18412 filed for tegra2 full failures) (kliegs investigating internal bot failures)' [16:47:27] *** reveman has joined #chromium-os [16:49:31] *** powderluv has joined #chromium-os [16:49:31] *** ChanServ sets mode: +v powderluv [16:49:48] <kliegs> powderluv: Are the full or binary builders supposed to upload prebuilts? [16:54:09] *** benchan has quit IRC [17:03:39] *** Soak has quit IRC [17:04:24] *** powderluv has quit IRC [17:09:58] *** JasonO has joined #chromium-os [17:12:18] <cmasone> kliegs: what bot failure are we talking about? x86-zgb-binary? [17:12:49] <kliegs> cmasone: i'm seeing different failures on pretty much every bot [17:13:03] <cmasone> kliegs: but that's the only failure [17:13:06] <kliegs> zgb binary had a chrome seg fault in cryptohome test [17:13:14] *** petermayo has joined #chromium-os [17:13:14] *** ChanServ sets mode: +v petermayo [17:13:14] <cmasone> kliegs: oh, unless the canaries? [17:13:16] <kliegs> what do you mean only failure? there are many red trees? [17:13:25] <cmasone> kliegs: yeah, but most don't matter [17:13:32] <kliegs> arm-tegra2 full, x86 pineview full are both red [17:13:44] <cmasone> kliegs: but you filed an issue for the former [17:13:56] <cmasone> so, it's chrome segv in zgb-binary, which is an understood failure mode [17:14:02] <cmasone> and then x86-pineview full [17:14:25] <kliegs> x86-mario-bin - failred to create a symlink for the remote_tests log link - I recall seeing a CL to fix a race condition on that [17:14:33] <kliegs> x86-pineview full I expect to go green real soon now [17:14:43] <kliegs> is x86-mario-release a closer? [17:15:07] <kliegs> and are the numbered builders closers on the internal tree? [17:15:37] <cmasone> kliegs: no [17:15:50] <cmasone> kliegs: my machine is horked, have to reboot [17:15:52] *** cmasone has quit IRC [17:23:04] <njw> back. sounds like we're not a whole lot clearer on the state of the world, eh? [17:23:17] <kliegs> njw: getting there. looks like multiple failiure modes across different bots [17:23:35] <kliegs> i'm generating a list of each bot and why its red [17:25:29] <kliegs> ok. so kaen/abel canaries are failing in initramfs - unable to find libgcc_s.so.1 [17:25:32] *** cmasone has joined #chromium-os [17:25:32] *** ChanServ sets mode: +v cmasone [17:25:41] <kliegs> cmasone: the *-canary's are closers? [17:25:53] <cmasone> kliegs: apparently [17:26:59] <kliegs> looks like a change to logging_CrashSender may have broken both alex & mario canaries. but the first broken build doesn't show a blame list [17:27:16] <kliegs> not sure if the test changed or if its catching an actual failure [17:27:55] <kliegs> I think the x86-mario-bin failure may have been fixed by another CL - I seem to recall a CL from sosa fixing a race condition on symlinking a directory for test log output which it looks like x86-mario-bin hit [17:28:44] <kliegs> x86-zgb-bin has the chrome segfault [17:29:39] <kliegs> and then the tegra2 builder has the prebuilt.py issue [17:43:34] *** wfrichar has joined #chromium-os [17:43:35] *** ChanServ sets mode: +v wfrichar [17:50:55] <crosbot> tree became 'Tree is closed (Bugs: x86-pineview-full: 18250, arm-tegra2-full: 18412, x86-{alex,zgb,mario}-canary: 18413, tegra2-{aebl,kaen}: 18414)' [17:55:47] <crosbot> tree became 'Tree is closed (Bugs: x86-pineview-full: 18250, arm-tegra2-full: 18412, x86-{alex,zgb,mario}-canary: 18413, tegra2-{aebl,kaen}: 18414 (CL in:http://gerrit.chromium.org/gerrit/4989)' [17:55:51] <kliegs> x86-zgb-bin went green. and ryanchag has a fix in for kaen/aebl [17:55:59] <kliegs> x86-pineview-full i still expect ot go green soon [17:56:25] <kliegs> so that leaves the tegra2-full prebuilt.py bug and the x86 crashSender bug [17:57:13] *** jrbarnette has joined #chromium-os [17:57:16] <kliegs> except tegra2 seaboard full just went red - checkin on red tree? [17:58:43] <kliegs> no - looks like flakiness fetching external package [17:58:59] <cmasone> o_O [17:59:07] <cmasone> fetching from teh outside? [17:59:56] <njw> I'm puzzling over the prebuilt.py thing a bit. The code looks like it's looking for a name target_variant or target-ignored, and tegra2 doesn't match that. (did it used to be arm-tegra2 somewhere?) [18:00:15] *** astro73|derrial has joined #chromium-os [18:00:51] *** arun_ has quit IRC [18:01:39] <crosbot> tree became 'Tree is closed (Bugs: x86-pineview-full: 18250, arm-tegra2-full: 18412, x86-{alex,zgb,mario}-canary: 18413, tegra2-{aebl,kaen}: 18414 (CL in:http://gerrit.chromium.org/gerrit/4989) , tegra2-seaboard-full: 18415 (looks like network flake))' [18:04:40] <kliegs> njw: yes - there's a CL from wednesday that changed the name [18:05:31] <kliegs> njw: http://gerrit.chromium.org/gerrit/4839 [18:05:54] *** astro73|derrial has left #chromium-os [18:06:04] *** stevenjb has joined #chromium-os [18:06:16] *** ChanServ sets mode: +v stevenjb [18:06:28] <sosa> yup totally me [18:06:41] <sosa> i'll revert that [18:06:45] <kliegs> sosa: thanks [18:06:58] <kliegs> sosa: Also - is clobbering broken? I meant to ask you last week but didn't have time to look into it [18:07:00] <sosa> not sure why prebuilt.py is so picky but i'll investigate [18:07:13] <kliegs> sosa: its got a regep that expects things in board_variant format [18:07:19] <kliegs> and has a special case for a few names that don't match that [18:07:28] <sosa> (it shouldn't be as far as I'm aware, nothing has changed in that pipeline but a few ppl have been touching cbuildbot so it's possible) [18:07:50] <kliegs> I didn't want to revert without checking on you as the change has been there for over a day so unsure of anything layered on it [18:08:09] <kliegs> sosa: you might be able to just modify prebuilt.py instead to allow tegra2 as a valid board name [18:08:20] <sosa> i'll revert [18:08:23] <kliegs> ok. thanks [18:08:24] <sosa> then change prebuilt.py [18:08:27] <sosa> then revert revert [18:08:28] <sosa> :) [18:08:46] <sosa> so i can do some testing and not rush to get a quick fix in in case something else is broken [18:08:48] <kliegs> :) Maybe we should just call those 'reapply'. I've seen a few CL's break my parser for if its applying or reverting [18:08:53] <kliegs> sounds good [18:09:07] <kliegs> also - can you help me as I've forgotten. do the binaries or full builders upload the prebuilts? [18:09:08] <sosa> yeah, re-apply :) [18:09:14] <sosa> both [18:09:28] <kliegs> ahh ok. I thought only binaries did so was confused [18:09:40] <sosa> kliegs: the binaries upload "temporary" ones that devs can use and the full's override those [18:09:57] <kliegs> ahh gotcha [18:09:58] <sosa> kliegs: that way devs don't have to build from source if they sync before a full run [18:10:05] <kliegs> also - clobbering? is it broken? [18:10:10] <kliegs> sosa: makes sense [18:10:10] <sosa> kliegs: shouldn't be [18:10:26] <stevenjb> Ah, nothing like red tree in the morning. kliegs / sosa, sounds like you are on top of it? [18:10:33] <sosa> kliegs: do you have an example of where you think one might have? [18:10:37] <kliegs> stevenjb: most of it. there are ~5 issues [18:10:55] <kliegs> sosa: It was on our local builder here so unsure of it was something wonky or not [18:11:01] <stevenjb> kliegs: Want to point me to something to investigate? [18:11:06] <kliegs> but I looked through the code and it looked like the clobber field got lost [18:11:20] <kliegs> stevenjb: gladly! Can you take crosbug.com/18413? [18:11:34] <kliegs> it looks like logging_crashSender test is failing on the alex/maria/zgb canaries [18:11:38] <kliegs> not sure who to track down on taht [18:11:52] <sosa> kliegs: is --clobber getting passed to cbuildbot on your local builder when you hit clobber? it may be a buildbot server code incompatibility or something [18:12:09] <kliegs> sosa: I mean within the cbuildbot code I didn't see the flag being passed to the right layers [18:12:14] <kliegs> so it wasn't doing the rm -rf [18:12:19] <sosa> kliegs: ah [18:12:24] <sosa> kliegs: let me do a quick test :) [18:12:31] <sosa> kliegs: locally of course [18:12:33] <sosa> :D [18:12:35] <kliegs> sosa :) [18:12:51] <kliegs> sosa: I had to clobber a red bot that had a stuck git repo which reminded me I was supposed to follow up on that [18:14:00] *** benchan has joined #chromium-os [18:14:00] *** ChanServ sets mode: +v benchan [18:14:06] <cmasone> kliegs: the Consent TO Send Stats thing is almost certainly related to 18317, 18352, and 16540 [18:14:42] *** mtennant has quit IRC [18:14:43] <sosa> kliegs: just ran cbuildbot -r /b/cbuild --buildbot --clobber x86-generic-pfq and the first thing it did (well not exactly the first) was rm -rf the root dir (/b/cbuild) [18:14:55] <cmasone> kliegs: it sounds like mkrebs is the right person [18:14:56] <stevenjb> kliegs: I'll take a look. Might take me a little bit to ramp up. [18:15:30] *** mtennant has joined #chromium-os [18:15:30] *** ChanServ sets mode: +v mtennant [18:15:39] <kliegs> stevenjb: read what cmasone just said above - that looks like it covers it. from 18352 it looks like the file is no longer used but the test is probably not updated to handle the new mechanism [18:16:18] *** jrbarnette_ has joined #chromium-os [18:16:18] *** ChanServ sets mode: +v jrbarnette_ [18:16:27] <kliegs> cmasone: thanks for the pointers [18:17:57] <stevenjb> kliegs - cool. I will see if I can ping mkrebs [18:18:20] <kliegs> sosa: ok good. probably something locally wrong. i'll look into it later unless petermayo beats me to it [18:19:21] *** jrbarnette has quit IRC [18:19:21] *** jrbarnette_ is now known as jrbarnette [18:23:13] <kliegs> stevenjb, njw: going to get some food. should be back in ~15 [18:23:27] <cmasone> stevenjb: mkrebs not here yet [18:23:47] <stevenjb> kliegs: ok, cool. Yeah, no response, I'll start poking around. [18:23:54] <kliegs> thanks [18:24:14] <cmasone> stevenjb: smells like one of those issues that only reproes on Official Builds :-/ [18:27:03] <stevenjb> oh joy [18:28:03] *** saintlou has joined #chromium-os [18:28:03] *** ChanServ sets mode: +v saintlou [18:35:22] *** powderluv has joined #chromium-os [18:35:22] *** ChanServ sets mode: +v powderluv [18:47:47] <stevenjb> kliegs / cmasone - looks like mario and alex canaries cycled green, so I am forcing a zgb build, see if that cycles green also. [18:49:30] *** powderluv has quit IRC [18:49:37] *** powderluv has joined #chromium-os [18:49:37] *** ChanServ sets mode: +v powderluv [18:55:31] <kliegs> pienview full is green too [18:56:09] <kliegs> and tegra2 should go green on next cycle after sosa's revert [18:56:45] <kliegs> zgb/kaen/abel canary's still red [18:59:31] <kliegs> stevenjb, njw: unfortunately need to duck out again - quick phonescreen. will be watching chat for anything critical [19:00:27] <stevenjb> kliegs: No worries. Since mario / alex canary went green, I am hoping for the best with the others, but it's a 4 hour cycle! [19:01:01] <stevenjb> I think that if tegra2 cycle green I should open the tree. [19:03:43] *** tbarzic__ has joined #chromium-os [19:04:45] <tbarzic__> I see the tree is nice and red for the start of the day :) [19:06:54] *** tbarzic has quit IRC [19:07:46] *** jrbarnette_ has joined #chromium-os [19:07:47] <stevenjb> tbarzic - yes, I think everything is cycling green. [19:08:10] <tbarzic__> oh, that's nice.. [19:10:14] *** tbarzic__ has left #chromium-os [19:10:41] *** tbarzic has joined #chromium-os [19:10:45] *** jrbarnette has quit IRC [19:12:26] *** jrbarnette_ has quit IRC [19:15:30] *** satorux_ has quit IRC [19:20:32] *** behdad has joined #chromium-os [19:25:57] *** JasonO has quit IRC [19:29:45] *** sbyer has quit IRC [19:31:41] *** powderluv has quit IRC [19:31:43] *** lipsinV2 has joined #chromium-os [19:32:39] *** sbyer has joined #chromium-os [19:32:39] *** ChanServ sets mode: +v sbyer [19:39:14] *** powderluv has joined #chromium-os [19:39:14] *** ChanServ sets mode: +v powderluv [19:40:44] *** rsleevi has joined #chromium-os [19:51:47] <njw> is the tegra2 full build actually running? [19:52:42] <stevenjb> njw: no [19:52:59] <stevenjb> I was just looking at that. [19:57:46] *** achuith has quit IRC [20:01:02] *** rharrison_chrome has quit IRC [20:03:46] *** JasonO has joined #chromium-os [20:03:47] *** JasonO has joined #chromium-os [20:03:51] <kliegs> sorry. back [20:04:22] <kliegs> zgb failed again? [20:04:24] *** achuith has joined #chromium-os [20:04:25] *** ChanServ sets mode: +v achuith [20:04:31] <stevenjb> kliegs: wb. I'm waiting on arm tegra2 seaboard full to complete to open the tree. [20:04:45] *** SoulRaven has quit IRC [20:04:49] <kliegs> i'm worried about the flakiness on zgb-binary - another chrome segfault [20:06:18] <kliegs> x86-zgb-canary you're expecting to go green given mario & alex did? [20:06:19] *** jrbarnette has joined #chromium-os [20:06:19] *** ChanServ sets mode: +v jrbarnette [20:06:20] <stevenjb> oh, zgb-binary. Yeah, I was trying to parse the failure output, but... ick. [20:06:32] <stevenjb> kliegs: s/expecting/hoping, but yeah. [20:06:39] <kliegs> and same for aebl canary as kaen went [20:06:47] <stevenjb> yep [20:06:52] <kliegs> yah. that seems reasonable [20:06:55] <stevenjb> 4 hour build cycle sucks though [20:07:23] <kliegs> flakiness on zgb-binary is worrying me - do you know if someone is actually looking into the crashes? or if its just going "hmm, flake, run again?" [20:07:34] <stevenjb> no idea. [20:07:43] <stevenjb> How are you finding the failure in the log? [20:08:18] <kliegs> search for 'chrome sig 11' in the Test section [20:09:09] <stevenjb> Ah, I see. Sigh. [20:09:22] <kliegs> yah. it had one red, one green, then red again [20:09:30] <kliegs> interesting, my chroot has 212 GB of data in it [20:09:33] <kliegs> wonder what that's about [20:09:44] <stevenjb> kliegs: Probably images [20:09:51] <kliegs> images aren't in the chroot [20:09:51] <stevenjb> I have a doc somewhere... [20:10:13] <kliegs> well - the built images aren't. I checked that directory [20:11:47] <kliegs> oh. multiple chrome checkouts in chroot/var/lib/portage/distfiles-target. each 20gb [20:12:39] <kliegs> so I'm not certain who's best for tracking down the chrome segfaults or how to go forward on that [20:13:16] <kliegs> tegra2 seaboard looks like it will be good - it passed the point it crashed [20:13:25] <stevenjb> yay [20:14:02] <kliegs> oh weird. i've been signed out of chat. but not gmail ... [20:19:36] <stevenjb> kliegs: FYI, some places to look for recovering disk space (although I think you found the largest offender): https://sites.google.com/a/google.com/chromeos/resources/engineering/chrome-for-chromeos-building-and-debugging#TOC-Recovering-Disk-Space [20:22:07] <crosbot> tree became 'Tree is closed (Bugs: x86-pineview-full: 18250, arm-tegra2-full: 18412, x86-{alex,zgb,mario}-canary: 18413, tegra2-{aebl,kaen}: 18414 (CL in:http://gerrit.chromium.org/gerrit/4989) , tegra2-seaboard-full: 18415 (looks like network flake))' [20:24:45] <stevenjb> cmasone: Any thoughts on why suite_Smoke/login_CryptohomeMounted is occasionally geting sig 11's? [20:24:55] <stevenjb> http://chromeos-botmaster.mtv.corp.google.com:8026/builders/x86-zgb-binary/builds/602/steps/Test/logs/stdio (search for 'chrome sig 11') [20:25:14] <stevenjb> or better, just search on 'sig 11' [20:25:38] <stevenjb> "login_CryptohomeMounted Received crash notification for chrome[9259] sig 11 (ignoring - chrome crash)" [20:25:55] <gauravsh> stevenjb: Without the crash dumps, it is really hard to tell [20:25:56] <stevenjb> kliegs: Is there an issue for the sig 11 failure? [20:25:57] <cmasone> stevenjb: it's the same intermittent chrome segv we've been seeing for a week. Chrome segfaults on logout sometimes. [20:25:57] <njw> crosbug.com/17899 would suggest that we have NFI what's going on here. [20:26:19] <stevenjb> njw: ok, thanks. [20:26:19] <cmasone> yes, and 17898 tracks work to get stack traces for chrome crashes during testing [20:26:27] <cmasone> which is the only way we're going to catch this [20:26:38] <kliegs> yah. need the core dump [20:26:41] <cmasone> I know [20:26:47] <kliegs> unless someone on chromium has magic insights [20:26:49] <cmasone> that's why someone is working on that [20:26:52] <cmasone> and has been for days [20:26:52] <kliegs> great [20:27:07] <stevenjb> ok, cool. So we can ignore that for now, good to know. [20:27:40] <kliegs> x86 generic pfq red? [20:28:22] <stevenjb> ugh, and I was just thinking we could open the tree up. [20:28:41] <kliegs> same [20:28:49] <kliegs> x86 generic pfq is showing the same failure mario/alex/... had [20:28:49] *** behdad has quit IRC [20:29:00] <kliegs> with Consent to Send Stats [20:29:06] <stevenjb> sigh [20:29:34] *** Soak has joined #chromium-os [20:29:37] <Soak> hi [20:30:07] <kliegs> stevenjb: can you follow up on that? You had more details than me on it from earlier [20:30:32] <Soak> it's ok I can choose my usb key for boot on Chromium OS, but when the PC boot on, there is a black screen.. [20:30:34] <stevenjb> kliegs: I've been poking at that, but haven't actually learned anything. [20:30:42] <Soak> I can type anything on, what I can do please? [20:30:45] <kliegs> hm. thought I saw someone mention a fix [20:32:31] <stevenjb> It either fixes itself, or is flakey. [20:33:40] <stevenjb> My guess, and this is just a guess, is that it may be an ordering dependency - it seems to be failing because "/home/chronos/Consent To Send Stats" is missing, so if another test is putting it there, future runs will succeed? [20:33:52] <crosbot> tree became 'Tree is closed (Bugs: arm-tegra2-full: 18412, x86-generic pfq/x86-zgb-canary: 18413, tegra2-aebl-canary: 18414 (CL in:http://gerrit.chromium.org/gerrit/4989) , tegra2-seaboard-full: 18415 (looks like network flake))' [20:34:03] *** behdad has joined #chromium-os [20:34:41] *** Soak has quit IRC [20:37:04] <kliegs> stevenjb: where you able to get ahold of mkrebs? [20:37:53] <stevenjb> No. I'm pinging kmixter who appears to have the bulk of the commits to logging_CrashSender.py, hoping he can help. [20:38:40] <kliegs> ok. [20:38:48] <kliegs> you might need to talk over to desks to catch them before lunch [20:39:02] <kliegs> hopefully either kmixter or mkrebs can help on this. would be nice to be able to open tree before lunch [20:39:27] <stevenjb> I'm in SF, but talking to Ken now [20:39:33] *** patcito has joined #chromium-os [20:39:55] <kliegs> oh. sorry. thought you were mtv [20:40:09] <kliegs> I need a remote robot I can wheel to cubes to find people [20:40:18] <gauravsh> i sit near him - what has Ken done now? [20:40:30] <gauravsh> is it about the status of the crash reporter bug? [20:40:54] <kliegs> yes. it just showed up on x86 generic pfq as well. [20:42:27] *** Soak has joined #chromium-os [20:42:46] <stevenjb> Talking to ken, it's mkrebs we need to get ahold of [20:43:01] <Soak> no one can help me please? Ctrl Alt F2 works I can login on chronos, but I have only a prompt.. [20:43:04] <kliegs> tbarzic: Can you wander by his cube? [20:43:05] <stevenjb> http://code.google.com/p/chromium-os/issues/detail?id=18413 [20:43:37] <kliegs> stevenjb: If we can't get mkrebs, does kmixter know a CL we can just revert? [20:43:40] <kliegs> And can we get him on irc? [20:44:07] <tbarzic> sure... [20:44:11] <kliegs> thanks [20:44:15] <stevenjb> kliegs: See comment by kmixter on 18413 [20:44:28] <tbarzic> what's his ldap [20:44:51] <stevenjb> mkrebs [20:48:41] <stevenjb> kliegs: It's got to be flakiness. I don't see anything in build 6694 that remotely points to the failure: http://build.chromium.org/p/chromiumos/builders/x86%20generic%20pre%20flight%20queue/builds/6694 [20:48:57] *** achuith has quit IRC [20:49:19] <tbarzic> mkrebs's not at his desk... [20:49:35] *** Soak has quit IRC [20:49:46] <tbarzic> and has not been seen today there [20:49:46] <kliegs> stevenjb: agreed. next build went green too [20:50:25] <kliegs> well. agreed 6994 had nothing that should have triggrered the bug. [20:50:38] <kliegs> And it seems likely to be flaky. kmixter can't help? [20:50:42] <stevenjb> So it did. So, open tree with a note on issue 18413 causing flakiness? [20:50:43] <kliegs> Does anyone know if mkrebs is working today? [20:51:13] <stevenjb> Nothing in his calendar. [20:51:23] <kliegs> There should really be someone else on that team who can look at this [20:51:37] <kliegs> Its failure rate seems high enough that reopening now means every other build could fail which isn't really a good state [20:53:29] <stevenjb> kmixter suggests disabling the test until 18413 gets fixed. [20:53:36] <stevenjb> He doesn't think mkrebs is in today. [20:54:06] *** achuith has joined #chromium-os [20:54:06] *** ChanServ sets mode: +v achuith [20:54:32] *** kmixter has joined #chromium-os [20:54:37] <kliegs> seaboard went green - one down [20:55:01] <crosbot> tree became 'Tree is closed (Bugs: arm-tegra2-full: 18412, x86-generic pfq/x86-zgb-canary: 18413, tegra2-aebl-canary: 18414 (CL in:http://gerrit.chromium.org/gerrit/4989) )' [20:55:55] <stevenjb> kliegs: What is the status of 18412? [20:56:16] <kliegs> so tegra2-full should go green on mason'es revert. x86-generic/x86-zgb will lose the flake if we disable the test. aebl should go green because kaen did [20:56:52] <kliegs> stevenjb: I expect 18412 to go green as a result of sosa's revert. sosa can you confirm that? [20:57:09] <kliegs> If we can get the test for crashsended disabled we should be fine to open tree [20:57:21] <kliegs> can you get kmixter in here? will be easier than making you relay everything [20:57:27] <kliegs> he didn't respond t me on chat [20:57:39] <stevenjb> kliegs: I think he just went to lunch. [20:58:00] <stevenjb> I'll look into disabling that test; should be easy enough? [20:58:03] <kliegs> yah [20:58:14] <kliegs> I'm just not sure on the exact policy/protocol/level of concern for disabling tests [20:58:31] <kmixter> i' here [20:58:55] <kmixter> no clue what's causing the Consent to send stats failure [20:59:03] <kmixter> it's not in the test, it's in the autotest framework that it's failing [20:59:18] <kmixter> so i can remove the test from smoke suite and let people who have looked at crash stuff lately diagnose it [20:59:47] <kliegs> kmixter: that should be good enough. if the test is flaking its not much use. and would mask/hinder finding other issues [20:59:58] <kliegs> do you think they'd be able to get in a fix today? or would it wait until monday? [21:00:28] <kmixter> i should have a review in the next few minutes [21:01:03] <stevenjb> I think that's best, given that otherwise the tree is likely to trend red over the weekend. We should just make sure to make 18413 a P0 and re-enable the test as soon as it's fixed. [21:01:21] <kliegs> stevenjb, kmixter: agreed [21:01:35] <stevenjb> I am sheriff Monday, so i can keep tabs on the status. [21:05:15] <kliegs> sounds good. soon as that fix from kmixter is in we can open [21:05:43] <kmixter> sent cl [21:07:11] <micahc> pwd [21:07:18] <kliegs> #chromium-os [21:07:29] <kliegs> kmixter: lgtm [21:09:00] <kmixter> merged [21:10:33] <kliegs> kmixter: thanks [21:11:07] <kliegs> x86 pineview binary fail on the chrome crash. this is really worrying me [21:11:37] <kliegs> i'm a bit nervous opening the tree with that flake. but it sounds like there's not much we can do [21:11:53] <kliegs> and we shouldn't really stop cros dev due to chromium bug. i'm probably just being overly paranoid [21:12:21] <kliegs> stevenjb, njw, tbarzic: Open it then? [21:12:28] <stevenjb> sgtm [21:13:33] <kliegs> i'll leave a pointer to the segfault bug in status message so people are aware of it [21:13:37] <stevenjb> kliegs: Chatting with cmasone, it seems like the chrome crash issue is being addressed, and flakiness is low enough to be tolerable in the meanwhile. [21:13:39] <njw> I feel like there should be some loud flashing siren to herald the opening, but otherwise, yeah. [21:13:50] <tbarzic> I'm for it [21:13:56] <kliegs> njw: the loud siren will herald the reclosing when the hordes come in [21:14:03] <stevenjb> no flashing siren. I'm already nervous about the incoming flood of commits... [21:14:16] <stevenjb> exactly. [21:16:47] <crosbot> tree became 'Tree is open (Known issues: crosbug.com/17899 - flaky chrome seg11 during logout in some test. ignore for now, WIP to resolve. arm-tegra2-full: crosbug.com/18412-expected to go green shortly on sosa revert, x86-zgb-canary: crosbug.com/18413-expected to go green on kmixter test disable, tegra2-aebl-canary: crosbug.com/18414 - expected to go green on rangchang cl)' [21:16:55] <kliegs> ok. as the full builders go green (probably after quitting time on this coast) lets remove the notes from the status message [21:17:13] <njw> I'm thinking something like this: http://www.youtube.com/watch?v=8RIfDYdzNIk&t=2m27s [21:17:15] <stevenjb> sg [21:17:16] <kliegs> i just don't want us to forget them. and to be able to notice if they go red for something different [21:17:27] <stevenjb> yep. [21:18:16] <kliegs> njw: Are we the happy vault dwellers in that clip? [21:18:49] <njw> clearly. [21:19:48] *** sergiu has joined #chromium-os [21:20:08] <ellyjones> please avoid being machine-gunned to death; I do not want to take over your sheriffing shift [21:20:24] <kliegs> ellyjones: aren't you the machine-gunner? [21:20:37] <ellyjones> I haven't committed anything today [21:20:48] <ellyjones> I'm still stuck in a hellish netherworld of interdependent changes to verity [21:21:24] <kliegs> watch out for grues [21:22:09] <ellyjones> I'm tempted to /w 2 [21:22:12] <ellyjones> drat. [21:23:32] <stevenjb> kliegs: Grabbing lunch. Shouldn't be gone too long. [21:27:05] <ellyjones> wfrichar: adlr: why does --run_as_root exist? [21:30:25] *** thieule_ has joined #chromium-os [21:30:52] *** lipsinV2 has quit IRC [21:33:20] *** espeed has joined #chromium-os [21:42:37] <wfrichar> hysterical raisins [21:42:46] <wfrichar> it serves no purpose. [21:43:00] <ellyjones> cool [21:43:02] <ellyjones> it always annoys me [21:43:08] *** behdad has quit IRC [21:43:15] <ellyjones> I will remove it [21:43:24] <wfrichar> I preemptively LGTM it. [21:43:53] <ellyjones> presumably no scripts rely on it [21:45:29] <wfrichar> platform/factory_installer/factory_install.sh, platform/initramfs/recovery_init.sh, platform/installer/chromeos-chroot-postinst [21:45:32] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "x86-alex-binary" from b01df17125a2273d5c685452c49162aafb582215: jennb at chromium dot org <jennb at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>, junov at chromium dot org <junov at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>)' [21:46:10] <wfrichar> you could just leave the argument in, but make it a noop [21:47:45] <ellyjones> yes [21:47:47] <ellyjones> I am doing that [21:48:22] <tbarzic> looks like chrome crash again [21:50:14] <stevenjb> joy. looking. [21:52:25] <crosbot> tree became 'Tree is open (Known issues: crosbug.com/17899 - flaky chrome seg11 during logout in some tests (x86-alex-binary). Fix in progress. arm-tegra2-full: crosbug.com/18412-expected to go green shortly on sosa revert, x86-zgb-canary: crosbug.com/18413-expected to go green on kmixter test disable, tegra2-aebl-canary: crosbug.com/18414 - expected to go green on rangchang cl)' [21:55:53] <ellyjones> 4007 root 20 0 130m 5768 1512 R 100 0.0 0:19.37 qemu-arm [21:55:56] <ellyjones> !? [21:59:07] <ellyjones> wfrichar: http://gerrit.chromium.org/gerrit/#change,5017 [22:00:01] <wfrichar> done [22:00:27] <ellyjones> ok [22:00:31] <ellyjones> still testing it, then I will push [22:02:27] *** behdad has joined #chromium-os [22:04:26] <ellyjones> there [22:06:14] <wfrichar> thank you! [22:12:35] *** jrbarnette has quit IRC [22:17:09] *** behdad has quit IRC [22:24:18] *** Koyktos has joined #chromium-os [22:26:14] *** rush2end_ has joined #chromium-os [22:27:53] *** rush2end has quit IRC [22:27:56] *** rush2end_ is now known as rush2end [22:41:01] *** behdad has joined #chromium-os [22:44:33] *** behdad has quit IRC [22:53:42] <crosbot> tree became 'Tree is open (Known issues: crosbug.com/17899 - flaky chrome seg11 during logout in some tests (x86-alex-binary). Fix in progress. tegra2-aebl-canary: crosbug.com/18414 - expected to go green on rangchang cl)' [22:58:53] <cmasone> stevenjb: we _should_ be collecting and symbolizing chrome crash dumps during tests now. At least on binary builders, full builds may not have picked up that change yet. [22:59:35] <stevenjb> cmasone: How can I comfirm this / look at the crash dumps? [23:00:04] <cmasone> stevenjb: from a run with crashes, get the test_results.tgz file [23:03:52] <stevenjb> cmasone: Pretend I've never done that before. I assume this is documented... somewhere? Specifically, how to get test_results.tgz from a buildbot run? [23:04:47] <cmasone> stevenjb: you go find the "BUILD ARTIFACTS ARE STORED AT" URL that's in the log of the Results stage [23:04:55] <cmasone> sorry , Report stage [23:05:14] <cmasone> then search in there for .dmp files, I forget exactly where in the tar they are and I really have to run right now :-) [23:05:58] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot_master" on "x86 generic pre flight queue" from d5aadf39424d736b836b375b0d634fae785caf98: Bill Richardson <wfrichar at chromium dot org>, Matt Tennant <mtennant at chromium dot org>, _third_party_ at chromium dot org, bsalomon at google dot com <bsalomon at google dot com@2bbb7eff-a529-9590-31e7-b0007b416f81>, mikhal at google dot com <mikhal at google dot com@4adac7df-926f-26a2-2b94-8c16560cd09d>, sergeyu at google dot com <sergeyu at google dot com@dd [23:06:20] <stevenjb> Oh, hey, that's handy. How difficult would it be to make that a link on the build summary page? (I'll go ahead and vote for or add that to the dory...) [23:06:46] <stevenjb> Hey, speak of the devil... [23:09:51] *** Inumedia has joined #chromium-os [23:10:33] <crosbot> tree became 'Tree is open (Known issues: crosbug.com/17899 - flaky chrome seg11 during logout in some tests (x86 generic pre flight queue). Fix in progress. tegra2-aebl-canary: crosbug.com/18414 - expected to go green on rangchang cl)' [23:16:56] <crosbot> tree became 'Tree is open (Known issues: crosbug.com/17899 - flaky chrome seg11 during logout in some tests (x86 generic pre flight queue). Fix in progress)' [23:16:58] <kliegs> did the chromiumos tree status page just gain color? [23:17:06] <kliegs> or has it been there all along and I never noticed? [23:17:52] <tbarzic> it seems it did [23:18:18] <tbarzic> i remeber it being black and white.. [23:19:06] *** wfrichar has quit IRC [23:23:47] *** wfrichar has joined #chromium-os [23:23:48] *** ChanServ sets mode: +v wfrichar [23:24:21] *** Koyktos has quit IRC [23:24:50] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "x86-zgb-binary" from e51b1906dcfaa253508bcceee105059ec32ab31c: tfarina at chromium dot org <tfarina at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>, wtc at chromium dot org <wtc at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>)' [23:25:12] *** FusionX has quit IRC [23:26:53] <crosbot> tree became 'Tree is open (Known issues: crosbug.com/17899 - flaky chrome seg11 during logout in some tests (x86 generic pre flight queue, x86 zgb binary). Fix in progress)' [23:29:40] *** FusionX has joined #chromium-os [23:32:28] *** rbyers has quit IRC [23:33:33] *** Ruetobas has quit IRC [23:40:22] *** Ruetobas has joined #chromium-os [23:46:52] *** vmil86 has quit IRC [23:47:56] *** kerz has left #chromium-os [23:48:20] *** kerz has joined #chromium-os [23:48:21] *** ChanServ sets mode: +v kerz