[00:00:22] <ellyjones> hrm [00:00:43] <eugeni> I've built chromium os at outlined at http://www.chromium.org/chromium-os/developer-guide, but when I boot the image either on real hw or within a VM, it gets to display the login window, but then reboots.. what is the best way to debug why it happens? [00:00:53] *** rharrison_chrome has quit IRC [00:01:12] <ellyjones> sosa: vm images are treated like usb boot - but you swap sdb3 for sda3? [00:02:32] <ellyjones> adlr: your built image is unbootable, right? [00:02:41] <adlr> ellyjones: i didn't try to boot it [00:02:43] <sosa> ellyjones: yeah [00:02:51] <ellyjones> adlr: could you? :) [00:03:28] <adlr> hm, let me see [00:03:34] <adlr> i don't run X usually... [00:05:56] <ellyjones> I am somewhat back at square 1 about my vm not booting, though [00:06:47] <cmasone> ellyjones: did you pass --dont-be-a-piece-of-shit-and-just-fucking-work [00:06:48] <cmasone> ? [00:06:50] <adlr> ellyjones: what VM do you use? [00:06:53] <cmasone> I usually forget that [00:07:17] <ellyjones> qemu [00:07:29] <adlr> what args? [00:07:29] <ellyjones> cmasone: can't tell if joking or serious [00:07:46] <ellyjones> adlr: the ones that image_to_vm emits: [00:07:46] <ellyjones> sudo kvm -m 1024 -vga std -pidfile /tmp/kvm.pid -net nic,model=virtio -net user,hostfwd=tcp::9222-:22 \ [00:07:50] <ellyjones> -hda /home/ellyjones/trunk/src/build/images/x86-alex/0.15.869.2011_08_09_1342-a1/chromiumos_qemu_image.bin [00:08:27] <ellyjones> "Booting from hard disk..." then nothing [00:08:41] <Hexxeh> is there any easy way of just disabling cryptohome? it's totally breaking non-guest logins for me [00:08:57] *** Styx has quit IRC [00:09:57] <ellyjones> Hexxeh: you could stub out those bits of cryptohomed and nobody else will notice [00:10:15] *** Styx has joined #chromium-os [00:10:20] <cmasone> Hexxeh: it's supposed to gracefully use scrypt if there's no TPM...I think. At least it did like...a year ago [00:10:48] <Hexxeh> apparently it doesn't :P login just fails as it would if your un/pw were wrong [00:10:57] <Hexxeh> chrome log reveals it's actually a cryptohomed error [00:11:02] <adlr> pci_add_option_rom: failed to find romfile "pxe-virtio.bin" [00:11:14] <cmasone> adlr: that always happens [00:11:23] <adlr> oh [00:11:25] *** JasonO has quit IRC [00:11:27] <ellyjones> aha [00:11:34] <ellyjones> after about 20 seconds of qemu runtime I got X [00:11:46] <ellyjones> and there's the cros splash screen [00:11:58] <adlr> okay, some X issue now [00:12:00] <adlr> one sec [00:12:10] <ellyjones> so I have a bootable image without my other change [00:12:35] *** JasonO has joined #chromium-os [00:13:10] <adlr> hazzah! it boots [00:13:35] <adlr> ellyjones: sorry, i can log in [00:13:40] <ellyjones> ok, so my change _did_ break it [00:13:47] <ellyjones> now the question is how to debug :P [00:14:30] <adlr> if it makes you feel any better, between remote X and qemu, my keyboard mapping is all wrong [00:15:42] * ellyjones console="ttyS0,115200" [00:16:00] <Hexxeh> do any of the builders build the dev-board? [00:19:48] <Hexxeh> ooh, wonder if i can create a new overlay variant for this and have it build me u-boot too... [00:27:29] <ellyjones> adlr: do you know where unpack_partitions.sh comes from? [00:29:38] <adlr> ellyjones: no, i looked a bit when you asked, but didn't find it [00:29:46] <ellyjones> ok [00:29:51] <ellyjones> code search can't find it either (!?) [00:29:58] <ellyjones> jrbarnette: I bet you know :) [00:31:17] <ellyjones> gerrit can't do text search? wtf [00:32:33] <adlr> looks like ti's crosutils/emit_gpt_scripts.sh [00:32:39] <adlr> but i'm not sure where crosutils lives [00:33:54] *** Styx has quit IRC [00:34:36] <crosbot> tree became '?' [00:34:51] <crosbot> tree became 'Tree is open (there are multiple crashers. One is the signall 11 in the login test 18269; the other is 17899; these don't need fioxed to open the tree. The other is the flash update, which does need fixed)' [00:35:55] <jrbarnette> ellyjones: take a look in src/scripts/build_library/build_gpt.sh [00:36:33] <jrbarnette> emit_gpt_scripts.sh is no longer used, and should be gone at this point [00:37:00] <ellyjones> ok, cool [00:37:21] <jrbarnette> the *pack_partitions scripts are generated automatically during build_image [00:37:43] <jrbarnette> the hard-coded offsets are specifically keyed to chromiumos_base_image.bin [00:37:56] <jrbarnette> they'll match the offsets in dev and test images, too [00:38:21] <jrbarnette> they won't necessarily match the offsets in any other image generated by install_gpt [00:38:33] <ellyjones> I still don't know how to debug this though :) I can't get serial output from qemu [00:38:39] <jrbarnette> or even other images created by build_image, if some partitions change size [00:38:58] <jrbarnette> oh, well, that's not a problem i can help with ;-) [00:39:19] <ellyjones> '-serial pty' doesn't seem to produce anything on the pty in question [00:39:29] <ellyjones> which implies that I'm hosted before we get to serial output at all [00:39:59] * ellyjones will try earlyprintk [00:39:59] <jrbarnette> if the image is toast at build, yes, you'll be hosed before the kernel is ever loaded [00:40:05] <Hexxeh> are there chrome binaries built with touchui by default like there are non-touchui ones? [00:40:09] <jrbarnette> 'cause the kernel might not even *get* loaded... [00:40:51] <jrbarnette> ellyjones: n.b. my last statement; you have no guarantee that the kernel will get loaded far enough for earlyprintk to matter [00:41:12] <jrbarnette> (sorry to rain on the party :-) ) [00:42:15] <ellyjones> it's true, but the breakage from my CL would happen after that [00:42:50] <ellyjones> the lack of any output _at all_ after "Booting from hard disk..." is really disturbing though [00:42:59] <ellyjones> but the image (before image_to_vm) booted on my alex [00:44:22] <jrbarnette> right, if image_to_vm is broken, it it could leave the kernel unable to load, so you've got to get that reliable first [00:44:26] <jrbarnette> but you knew that :-) [00:44:39] <ellyjones> indeed, and image_to_vm produces a bootable image without my change present [00:44:42] <ellyjones> (my change is to the kernel) [00:45:00] <jrbarnette> oh, then you're probably set to try earlyprintk [00:45:05] <ellyjones> yep [00:45:11] <ellyjones> if that doesn't help... I will be sad :P [00:45:32] <jrbarnette> ... you're not sad already? :-P [00:45:34] <ellyjones> no change, no vm: boots. no change, vm: boots. change, no vm: boots. change, vm: doesn't boot [00:45:39] <ellyjones> well, I'm annoyed, but not sad yet :P [00:51:06] <jrbarnette> ellyjones: i just finished reading image_to_vm.sh; the bug is in fact glaring once you know what to look for [00:51:16] <jrbarnette> i know the fix; is anyone working on it? [00:51:32] <ellyjones> jrbarnette: no, do tell? [00:51:34] <ellyjones> (which bug?) [00:51:52] <jrbarnette> the one that prevents image_to_vm.sh from creating a bootable image [00:52:29] <jrbarnette> oh, wait, i may be ahead of myself [00:52:57] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot_master" on "x86 generic PFQ" from 9b02d49ae26dad133ed8fb11035dd9593cc4bb1d: jvoung at google dot com <jvoung at google dot com@fcba33aa-ac0c-11dd-b9e7-8d5594d729c2>)' [00:53:18] <ellyjones> a bug in image_to_vm?? [00:54:12] <jrbarnette> didn't you say it was corrupting partition 12? [00:54:17] <ellyjones> no [00:54:24] <ellyjones> that was unpack_partitions unpacking some random junk [00:54:24] <jrbarnette> anyway, the problem i thought i saw was a mirage [00:56:36] *** oc80z has quit IRC [00:57:04] *** oc80z has joined #chromium-os [01:04:25] *** snickersnack has quit IRC [01:04:54] <sosa> this chrome crasher seems to have gotten worse [01:05:56] <sosa> it's closing the tree almost every build run [01:06:00] <crosbot> tree became '?' [01:06:15] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot_master" on "x86 generic PFQ" from 9b02d49ae26dad133ed8fb11035dd9593cc4bb1d: jvoung at google dot com <jvoung at google dot com@fcba33aa-ac0c-11dd-b9e7-8d5594d729c2>)' [01:06:55] <sosa> sosa: I'd hate to say this, but perhaps should we temporarily disable error'ing out on sig 11's on that test? [01:07:29] *** snickersnack has joined #chromium-os [01:10:53] <cmasone> sosa: no, we shouldn't [01:13:05] <ellyjones> internal pfq 3948 is the first chrome sig 11, I believe [01:13:31] <sosa> cmasone: aight [01:16:29] <ellyjones> 14.0.830.0_rc-r1 is the first build with a sig 11 [01:16:42] <ellyjones> but we're missing logs for builds before 3943 :( [01:17:24] <cmasone> ellyjones: can you post that in 17899? at least it'll give an endpoint on the CLs to investigate [01:17:25] <ellyjones> sosa: do you know where I can get at the stdio logs for builds before 3943? [01:17:40] <ellyjones> cmasone: I'm not sure of it, though, because it's only 5 builds after the first log I have [01:17:50] <ellyjones> which may mean it happened earlier and I didn't see it [01:17:57] <sosa> ellyjones: it is possible they may have been cleaned up [01:18:00] <ellyjones> the next failure after 3948 is 3953 [01:18:01] <sosa> ellyjones: lemme check on the master [01:18:05] <ellyjones> thanks [01:19:15] *** FatDarrel has joined #chromium-os [01:19:24] <ellyjones> (3943 is Jul 22) [01:20:30] <ellyjones> ooh, but the zgb-canary logs still live [01:24:41] <ellyjones> 543 fails with chromeos-chrome-14.0.827.0_rc-r1, 542 passes with chromeos-chrome-14.0.826.0_rc-r1 [01:24:54] <ellyjones> cmasone: do you know how to turn the 826 into a chrome revision? [01:25:51] <sosa> logs aren't on the master, looks like they got cleaned up [01:26:00] <ellyjones> argh, oh well [01:26:03] <ellyjones> I will use the zgb ones [01:26:11] <ellyjones> sosa: do you know how to turn 826 into a chrome revision? [01:26:12] <sosa> 826, check the deps file [01:26:26] <sosa> http://src.chromium.org/viewvc/chrome/releases/ [01:27:01] <ellyjones> ok, so 92995 was healthy, 93179 was not [01:27:44] <crosbot> tree became 'Tree is open (More issue 17899; I've updated information in the bug; it looks like a race in the MessagePump teardown/task Run code)' [01:28:07] <ellyjones> then it was not seen again until 839 @ 94841 [01:29:48] <sosa> tlambert just updated the bug: http://code.google.com/p/chromium-os/issues/detail?id=17899 [01:29:50] * ellyjones adds info to the bug [01:29:52] <ellyjones> yeah, I saw :) [01:30:19] <sosa> yay progress [01:33:30] <tlambert> This really needs someone one the chrome side to put in canary checking, even if it's later ripped out. [01:33:40] <tlambert> ^one^on [01:33:59] <ellyjones> I added what I know to the bug [01:34:02] <ellyjones> now I'd better head home [01:34:03] <ellyjones> later all :) [01:34:08] <adlr> night [01:43:01] *** Keybuk has quit IRC [01:56:33] *** powderluv has joined #chromium-os [01:56:33] *** ChanServ sets mode: +v powderluv [02:04:38] *** oc80z has quit IRC [02:05:33] *** oc80z has joined #chromium-os [02:18:47] *** oc80z has quit IRC [02:18:47] *** oc80z has joined #chromium-os [02:31:39] *** bpontes has quit IRC [02:32:06] *** chocobo__ has quit IRC [02:34:13] *** maruel has quit IRC [02:43:03] <dhendrix> What does CHROMEOS_RELEASE_NAME from /etc/lsb-release on a mass production / retail machine display? [02:45:30] <vlaviano> "Chrome OS" [02:45:50] <dhendrix> vlaviano: thanks! [02:46:43] *** stevenjb has quit IRC [02:54:32] *** saintlou has quit IRC [02:54:57] *** saintlou has joined #chromium-os [02:59:08] *** saintlou has quit IRC [03:07:31] *** jrbarnette has quit IRC [03:09:00] *** wfrichar has quit IRC [03:10:53] *** grundler has quit IRC [03:31:47] *** lipsinV2 has joined #chromium-os [03:35:38] <sergiu> any ideas why build-package fails on arm with this: [03:35:44] <sergiu> /usr/libexec/gcc/armv7a-cros-linux-gnueabi/ld: cannot find -lLmiAudioCommon [03:35:45] <sergiu> collect2: ld returned 1 exit status [03:35:57] <sergiu> it's gtalk that fails [03:36:07] <sergiu> building arm-generic [03:37:41] *** oc80z has quit IRC [03:38:40] *** oc80z has joined #chromium-os [03:46:11] *** lipsinV2 has quit IRC [03:48:23] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "x86 pineview full" from 606b73a4cf02bf1279971aee9cb5e98a85289554: Nick Sanders <nsanders at chromium dot org>, Simon Que <sque at chromium dot org>, _third_party_ at chromium dot org, mseaborn at chromium dot org <mseaborn at chromium dot org@fcba33aa-ac0c-11dd-b9e7-8d5594d729c2>, pasko at google dot com <pasko at google dot com@fcba33aa-ac0c-11dd-b9e7-8d5594d729c2>)' [03:54:24] *** sadrul has joined #chromium-os [03:54:24] *** ChanServ sets mode: +v sadrul [03:56:50] <crosbot> tree became 'Tree is open (same problem: crosbug.com/17899)' [03:59:24] *** lipsinV2 has joined #chromium-os [04:04:13] *** Kernel-Panic has quit IRC [04:09:29] *** rbyers_ has joined #chromium-os [04:09:44] *** rbyers_ has quit IRC [04:10:41] *** rbyers_ has joined #chromium-os [04:13:39] *** Kernel-Panic has joined #chromium-os [04:16:46] <srao> sergiu: not sure if gtalk will build on arm-generic [04:17:11] <srao> sergiu: have you built that before ? [04:20:50] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "x86-alex_he canary" from None: )' [04:21:25] *** lipsinV2 has quit IRC [04:23:02] *** lipsinV2 has joined #chromium-os [04:23:22] *** Kernel-Panic has quit IRC [04:23:45] *** Kernel-Panic has joined #chromium-os [04:25:09] <crosbot> tree became 'Tree is open (same problem: crosbug.com/17899)' [04:25:35] *** rbyers_ has quit IRC [04:50:27] *** BladeFreak has quit IRC [04:55:25] *** kerz has quit IRC [05:03:29] *** powderluv has quit IRC [05:12:50] *** JasonO has quit IRC [05:22:13] <sergiu> srao, no, I'll try a more specific build, maybe that works [05:35:46] *** Kernel-Panic has quit IRC [05:49:22] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot_master" on "TOT Pre-Flight Queue" from 18ca15a92ffbdbc3381500bb13ac57d26b8d21d0: thakis at chromium dot org <thakis at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>)' [06:09:06] *** FusionX has quit IRC [06:17:02] *** shaggymane has joined #chromium-os [06:20:17] <shaggymane> did chroot just get updated? [06:23:01] *** D|sT has quit IRC [06:26:04] <sergiu> anyone with a x86-generic still at work? :D [06:32:58] *** sergiu has quit IRC [06:39:47] *** achuith has quit IRC [06:39:57] *** achuith has joined #chromium-os [06:39:57] *** ChanServ sets mode: +v achuith [07:17:06] *** shaggymane has quit IRC [07:18:49] *** vmil86 has joined #chromium-os [07:21:14] *** saintlou has joined #chromium-os [07:22:04] *** saintlou has quit IRC [07:22:08] *** saintlou has joined #chromium-os [07:22:09] *** ChanServ sets mode: +v saintlou [07:28:51] *** superm1 has quit IRC [07:28:56] *** superm1 has joined #chromium-os [07:28:57] *** superm1 has joined #chromium-os [07:55:48] *** yusukes_ has joined #chromium-os [08:17:38] *** cooled has quit IRC [08:34:03] *** shaggymane has joined #chromium-os [08:34:18] *** cooled has joined #chromium-os [08:44:28] *** JakeSays has quit IRC [08:48:46] *** powderluv has joined #chromium-os [08:48:47] *** ChanServ sets mode: +v powderluv [08:56:11] *** unreal has quit IRC [08:58:39] *** SoulRaven has joined #chromium-os [08:59:15] *** Styx has joined #chromium-os [09:03:22] *** JakeSays has joined #chromium-os [09:06:33] *** cooled has quit IRC [09:15:25] *** unreal has joined #chromium-os [09:43:31] *** cooled has joined #chromium-os [10:10:41] *** saintlou has quit IRC [10:25:00] <crosbot> tree became 'Tree is open (same problem: crosbug.com/17899)' [10:44:43] *** patcito has quit IRC [10:54:37] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "x86-alex canary" from None: )' [11:03:11] *** ScriptRipper has quit IRC [11:04:40] *** ScriptRipper has joined #chromium-os [11:32:22] *** sadrul has quit IRC [11:51:24] *** echelog-1` has joined #chromium-os [11:51:39] *** Guest54618 has quit IRC [11:51:39] *** Inumedia has quit IRC [11:51:47] *** |simo| has joined #chromium-os [11:52:04] *** echelog-1` is now known as echelog-1 [11:52:04] *** echelog-1 has quit IRC [11:52:27] *** echelog-1 has joined #chromium-os [12:08:17] *** jochen__ has quit IRC [12:08:23] *** jochen__ has joined #chromium-os [12:08:24] *** ChanServ sets mode: +v jochen__ [12:26:14] *** |simo- has joined #chromium-os [12:32:07] *** nick761 has quit IRC [12:32:07] *** Guest30849 has quit IRC [12:32:07] *** arun_ has quit IRC [12:32:07] *** Kyngdom has quit IRC [12:32:08] *** steev_ has quit IRC [12:32:08] *** eggie has quit IRC [12:32:08] *** dubroy has quit IRC [12:32:08] *** yosafbridge has quit IRC [12:32:09] *** zbehan has quit IRC [12:32:09] *** nick761 has joined #chromium-os [12:35:24] *** yosafbridge has joined #chromium-os [12:38:28] *** steev_ has joined #chromium-os [12:38:38] *** arun_ has joined #chromium-os [12:41:40] *** zbehan has joined #chromium-os [12:43:22] *** Kyngdom has joined #chromium-os [12:46:20] *** eggie has joined #chromium-os [12:54:30] *** SoulRaven has quit IRC [12:59:27] *** SoulRaven has joined #chromium-os [13:14:32] *** arun has quit IRC [13:18:19] *** arun has joined #chromium-os [13:18:19] *** arun has quit IRC [13:18:19] *** arun has joined #chromium-os [13:49:03] *** SoulRaven has quit IRC [13:50:18] *** FusionX has joined #chromium-os [14:24:50] *** TW1920 has joined #chromium-os [14:27:56] *** nick761_ has joined #chromium-os [14:29:48] *** nick761 has quit IRC [14:35:28] *** arun has quit IRC [14:37:40] *** arun has joined #chromium-os [14:37:40] *** arun has joined #chromium-os [14:44:27] *** behdad has joined #chromium-os [14:48:44] *** rharrison_chrome has joined #chromium-os [14:48:44] *** ChanServ sets mode: +v rharrison_chrome [15:13:40] *** kelonye has joined #chromium-os [15:28:27] *** behdad1 has joined #chromium-os [15:28:33] *** behdad has quit IRC [15:31:54] *** behdad1 has quit IRC [15:32:06] *** behdad has joined #chromium-os [15:36:12] *** kelonye has left #chromium-os [15:36:57] *** sadrul has joined #chromium-os [15:36:57] *** ChanServ sets mode: +v sadrul [15:44:38] *** oc80z has quit IRC [15:44:39] *** oc80z has joined #chromium-os [16:02:36] *** flackr has quit IRC [16:03:30] *** D|sT has joined #chromium-os [16:03:51] *** saintlou has joined #chromium-os [16:03:51] *** ChanServ sets mode: +v saintlou [16:04:13] *** rbyers_ has joined #chromium-os [16:04:26] *** saintlou_ has joined #chromium-os [16:04:26] *** ChanServ sets mode: +v saintlou_ [16:05:46] *** behdad has quit IRC [16:05:46] *** stalled has quit IRC [16:06:35] *** behdad has joined #chromium-os [16:08:21] *** saintlou has quit IRC [16:11:50] *** arun has quit IRC [16:12:14] *** oc80z has quit IRC [16:13:40] *** oc80z has joined #chromium-os [16:14:05] *** arun has joined #chromium-os [16:16:16] *** stalled has joined #chromium-os [16:18:18] *** petermayo has joined #chromium-os [16:18:18] *** ChanServ sets mode: +v petermayo [16:20:54] *** oc80z has quit IRC [16:20:54] *** oc80z has joined #chromium-os [16:26:03] <ellyjones> crosbot: tree? [16:26:04] <crosbot> ellyjones: tree: Tree is closed (Automatic: "cbuildbot" on "x86-alex canary" from None: ) [16:26:09] <ellyjones> crosbot: sheriffs? [16:26:09] <crosbot> ellyjones: sheriffs: dlaurie, sleffler, zork, rochberg [16:32:46] <redpig> ellyjones: nice [16:33:33] *** simon____ has joined #chromium-os [16:34:00] *** stalled has quit IRC [16:34:02] <ellyjones> I will ask rochberg to look into it when he gets in [16:35:39] <ellyjones> why do the canary bots exist, anyway? (as distinct from the regular bots) [16:38:14] *** flackr has joined #chromium-os [16:38:18] *** Zach has joined #chromium-os [16:44:46] *** stalled has joined #chromium-os [16:48:06] *** jrbarnette has joined #chromium-os [16:49:10] *** cros_ has joined #chromium-os [16:49:13] <rochberg> I am in [16:49:23] <ellyjones> hiya [16:51:39] <cros_> reveman: Any thoughts on my questions regarding the window manager and my open GL ES implementation? [16:53:00] <reveman> cros_: I'm still confused about what you're trying to do [16:54:29] *** Zach has left #chromium-os [16:55:05] <cros_> I am trying to bring up chromium on a new SoC using open GL ES. Using my graphics driver and open GL ES implementation I am having UI problems. I see 4 login windows side by side so I am trying to figure out what is wrong. [16:56:26] <rochberg> x86-generic is another chrome segfault [16:58:34] *** stalled has quit IRC [17:00:10] *** saintlou_ has quit IRC [17:00:43] *** rbyers_ has quit IRC [17:01:03] <reveman> cros_: what graphics driver is the X server using? [17:01:28] *** oc80z has quit IRC [17:02:24] *** oc80z has joined #chromium-os [17:02:47] <cros_> reveman: fbdev [17:03:32] *** saintlou has joined #chromium-os [17:03:43] *** saintlou has joined #chromium-os [17:03:44] *** ChanServ sets mode: +v saintlou [17:04:33] *** saintlou has quit IRC [17:04:58] <rochberg> x86-mario canary has the standard chrome crash, but also an error running login_CryptohomeUnmounted. Is there any way I can get more data about that failure? http://chromeos-botmaster.mtv.corp.google.com:8026/builders/x86-mario%20canary/builds/844/steps/Test/logs/stdio [17:05:42] <rochberg> Search keyword "06:54:51" [17:07:15] <cros_> reveman: For 2D we just use the X11 standard software solution, no hardware [17:07:25] <cros_> reveman: We are only using the GPU hardware to do composition [17:10:33] <cmasone> rochberg: yeah, download the test logs. There's a link in the Report phase [17:11:16] <reveman> cros_: sounds like you're trying to run cros and not just chromium, correct? [17:11:17] *** stalled has joined #chromium-os [17:11:51] <rochberg> Separate question: What about turning off fail-on-segfault until 17899 is closed? [17:12:07] <cmasone> rochberg: already asked and rejected. [17:13:28] <cros_> reveman: Correct, chromium OS. If we build chromium OS to use software open GL then the chromium UI comes up fine. [17:14:32] *** jrbarnette_ has joined #chromium-os [17:14:32] *** ChanServ sets mode: +v jrbarnette_ [17:15:53] <reveman> cros_: so you're using the GLES backend for chromeos-wm? I don't know how well that works. I would not be surprised if it is broken. The GLX backend is normally used [17:17:13] <rochberg> URL for this conversation? Casual search of my mail is failing [17:17:31] <ellyjones> rochberg: it happened on IRC [17:18:37] *** jrbarnette has quit IRC [17:18:38] *** jrbarnette_ is now known as jrbarnette [17:20:33] *** Ruetobas has quit IRC [17:21:35] <rochberg> Casual search of echelog is failing too. I guess we'll have the same debate again on the mailing list then. [17:21:48] <ellyjones> cool [17:22:59] <ellyjones> I think the rationale is that it will decrease the pressure to fix 17899 [17:23:24] <cros_> reveman: Yes, we are using the GLES backend for chromeos-wm (UI composition). I see that is the default for the arm-generic target too so I would hope that works. [17:23:24] <cmasone> rochberg: why? The downside risk is that more segv's sneak in while we're not looking and then we trip over them later. The upside is that sheriffs don't have to be diligent. [17:24:07] <rochberg> I'm currently searching EBay for "Master Alarm Silence" buttons because the tree is so red that I'm going to spend the next hour just going from builder to builder triaging things. [17:25:55] <cmasone> rochberg: PFQ: chrome sig11; x86-generic-full ditto [17:26:20] *** 31NAAURFA has joined #chromium-os [17:26:21] <rochberg> Most of them are 17899, but not all. And if there's another chrome segfault that's snuck in, my current level of diagnosis sure isn't going to catch it [17:26:21] *** 92AADTP01 has joined #chromium-os [17:26:44] <cmasone> rochberg: x86-pineview-full: git hang [17:27:03] <cmasone> rochberg: do what you can [17:27:47] <crosbot> tree became 'Tree is closed. rochberg investigating x86-mario canary' [17:29:06] *** dlaurie has joined #chromium-os [17:29:06] *** ChanServ sets mode: +v dlaurie [17:29:46] <cmasone> x86-alex-binary failed in the UploadPrebuilts step [17:30:51] <cmasone> that should probably be escalated to a trooper [17:31:04] <cros_> Has anyone tested or verified that the Open GL ES UI composition works on any ARM platform? [17:31:42] *** mascondante has joined #chromium-os [17:31:57] *** stalled has quit IRC [17:32:53] <crosbot> tree became 'Tree is closed. rochberg investigating x86-mario canary, x86-alex-canary -> cmasone' [17:34:39] <reveman> cros_: how is your driver implementing the EGL_KHR_gl_image extension? [17:34:48] *** behdad has quit IRC [17:35:05] <reveman> cros_: have you verified that that extension works properly? [17:36:16] <cros_> reveman: We verified it using glcompbench benchmark from Linaro. [17:37:57] <cmasone> rochberg: I know what happened to x86-alex-canary. We upped the timeout on chrome exiting when it's dumping a crash report, but did not also up the timeout on the test harness _waiting_ for it to log out [17:38:14] <cmasone> that led to all sorts of weirdness in the test harness on that bot [17:38:16] <cmasone> filing a bug [17:38:49] <rochberg> Thanks [17:38:56] <dlaurie> zgb-canary failure looks like cryptohomemounted crash as in 17899 [17:39:40] *** grundler has joined #chromium-os [17:39:40] *** ChanServ sets mode: +v grundler [17:42:20] <cmasone> http://code.google.com/p/chromium-os/issues/detail?id=18966 [17:42:41] <cros_> Are there any chromium OS window manager design documents available? [17:42:44] <cmasone> I would not consider this a bug to keep the tree closed, but I am starting to work on it [17:43:02] <cros_> Or any chromium OS window manager porting documents? [17:43:18] *** stalled has joined #chromium-os [17:46:10] *** sadrul has quit IRC [17:49:16] *** wfrichar has joined #chromium-os [17:49:23] *** ChanServ sets mode: +v wfrichar [17:50:07] *** behdad has joined #chromium-os [17:51:56] <gsam_> hmm, just in time for build breakage [17:52:09] <ellyjones> yep [17:52:38] <cmasone> testing a fix for the transient alex canary failure [17:54:15] <ttuttle|work> ellyjones: fun fact: if you do a fresh install, SafetyDance fails because Favorite=False on Verizon, which prevents it from clearing AutoConnect >.< [17:54:52] <ellyjones> ttuttle|work: fun fact 0 = 1 | n = n * fact (n - 1) [17:55:03] <ellyjones> ttuttle|work: that is unsurprising to hear, you should fix it [17:56:49] <cmasone> gsam_: rochberg dlaurie pinged the troopers+davidjames about the UploadPrebuilts failure on x86-alex-binary [17:57:00] <gsam_> ok [17:57:07] <gsam_> just trying to get up to speed [17:58:47] *** bpontes has joined #chromium-os [17:58:47] *** ChanServ sets mode: +v bpontes [17:58:57] <shaggymane> git question here... computer crashed now my git is messed and sync isnt working correctly how do I fix it? [18:00:15] <shaggymane> I dont want to re-dload all chromiumos again [18:00:16] *** saintlou has joined #chromium-os [18:00:17] <rochberg> x86-mario looks like a system crasxh [18:00:25] <ellyjones> in what manner is your git 'messed'? [18:00:46] <shaggymane> it got extra files and some deleted [18:01:15] <cmasone> shaggymane: this isn't really a git support forum... [18:01:26] <shaggymane> git is new ;( [18:01:30] <shaggymane> to me [18:01:31] <vpalatin> shaggymane: git reset --hard [18:01:41] <shaggymane> thankyou ! [18:02:08] <vpalatin> shaggymane: this will remove your local changes if you are developping smthing [18:02:56] <shaggymane> ty, was backing stuff up and harddrive dropped on the floor ... [18:03:18] <shaggymane> great backup [18:04:27] <ttuttle|work> ellyjones: I am thinking about writing a magic wand that gets 3G into a known state. [18:04:45] <ttuttle|work> ellyjones: (favorite=True, autoconnect=False, device powered off) [18:05:15] <ellyjones> go for it [18:05:24] <ellyjones> those are called 'functions' in the real world [18:06:42] *** saintlou has quit IRC [18:07:40] <gsam_> is the zgb-binary test failure known? [18:07:59] <gsam_> looks to be happening repeatedly [18:08:34] * gsam_ scans back [18:08:42] <cmasone> gsam_: isn't that 17899 [18:08:51] <cmasone> gsam_: or 18269 [18:09:26] *** redpig has quit IRC [18:11:02] <dlaurie> rochberg: x86-mario also looks to have the chrome crash during cryptohomemounted, is this another case of 17899? [18:12:04] <rochberg> I'm still looking at the whole-machine crash on x86-mario canary 844 [18:12:57] * gsam_ reads 17899 issue [18:13:25] <gsam_> i wouldn't be surprised if the crasher is also what i keep hitting in day-to-day use on r15 [18:14:25] <rochberg> cmasone: If the pre-crash sysinfo/var/log/messages ends with nothing interesting and the next test's messages start with a boot, are there any other diagnostics to be had? [18:15:07] <cmasone> rochberg: need more context. diagnostics for what? What do you mean "nothing itneresting"? [18:20:17] <rochberg> My reasoning is totally bogus. Never mind. [18:22:19] *** jrbarnette_ has joined #chromium-os [18:24:06] <gsam_> who is doing what for these failures and what can i do to help? [18:24:10] *** sadrul has joined #chromium-os [18:24:21] *** ChanServ sets mode: +v sadrul [18:24:35] <gsam_> (other than what status says) [18:25:16] *** jrbarnette has quit IRC [18:26:20] *** jrbarnette_ has quit IRC [18:26:50] <dlaurie> x86-alex-binary and x86-zgb-binary both just failed tests again with what looks like 17899 [18:27:09] <gsam_> dlaurie, yes [18:27:46] <cmasone> gsam_: get dave or zel to assign an engineer to 17899? [18:28:03] <gsam_> cmasone, ok, no dave here yet [18:28:06] <rochberg> I'm currently going through IRC to see what's still outstanding [18:28:22] <cmasone> I already posted a CL to address the (transient) x86-alex canary fail [18:28:31] <cmasone> wait [18:28:32] <cmasone> no I didn [18:28:33] <cmasone> t [18:28:49] <cmasone> stupid repo [18:29:03] <cmasone> http://gerrit.chromium.org/gerrit/#change,5660 [18:29:05] <cmasone> now I did [18:30:11] *** Keybuk has joined #chromium-os [18:30:11] <gsam_> cmasone, your cl looks to be noop [18:30:12] *** ChanServ sets mode: +v Keybuk [18:30:15] <rochberg> x86-mario-canary still has me confused, but I need to eat to continue. Back in 20. [18:30:35] <dlaurie> yeah the timeout is still 30, but it has a helpful comment to go with it [18:32:34] <cmasone> dlaurie: shit [18:32:56] <cmasone> dlaurie: last-minute change that I didn't vet [18:34:54] *** lipsinV2 has quit IRC [18:34:59] <Keybuk> morning! [18:35:47] <cmasone> dlaurie: PTAL [18:36:15] <dlaurie> ugh I can only +1, my account is messed from GA+ still, bug filed :/ [18:37:00] <cmasone> dlaurie: ok, I'll self-approve and push, shall I? [18:37:07] <dlaurie> yeah go for it [18:40:15] <shaggymane> Chromium OS Developer Guide - The Chromium Projects <http://www.chromium.org/chromium-os/developer-guide> devguide changed [18:41:47] *** mascondante has quit IRC [18:41:57] <shaggymane> ./enter_chroot.sh has changed to cros_sdk [18:43:40] <cmasone> this mario-canary issue is weird. It looks like time jumped. Like the VM was stopped for 20+ seconds [18:44:15] <cmasone> I see cryptohomed starting to mount the user's cryptohome (as a part of the login process) at 13:54:16 in /var/log/messages [18:44:37] <cmasone> that looks like the right timeframe; I saw login be initiated a few seconds prior in the autotest log [18:45:07] <cmasone> then nothing appears in any log until 54:42, which puts us right at the login timeout, so the test harness gives up [18:47:37] <rochberg> VM descheduled? [18:47:56] <gsam_> cmasone, how do i get /var/log/messages? [18:48:39] <rochberg> For the mario thing? http://chromeosbuild7.mtv.corp.google.com/archive/x86-mario-release/0.15.877.0-a1-b844 [18:48:46] <gsam_> ah, thanks [18:49:09] <rochberg> (linked to from the "report" stage) [18:49:26] *** jrbarnette has joined #chromium-os [18:50:00] <gsam_> i see thanks [18:52:35] *** Kernel-Panic has joined #chromium-os [18:54:41] <cmasone> rochberg: yeah, I'm wondering. Is that a thing that can happen? for that long? [18:54:47] *** stevenjb has joined #chromium-os [18:54:47] *** ChanServ sets mode: +v stevenjb [18:56:53] <crosbot> tree became 'Tree is closed. rochberg investigating x86-mario canary' [18:57:13] <cmasone> so, is there anything we feel can be done about this? [18:57:40] <gsam_> cmasone, i ping'd zel but no response [18:57:41] <cmasone> and is this the only thing that we're blocking on? [18:57:52] <cmasone> gsam_: shit, zel is in the air. I forgot. [18:58:04] <cmasone> gsam_: and dave is sheriffing from home [18:58:05] <gsam_> that explains that [18:58:30] <gsam_> cmasone, wouldn't he be on channel? [18:58:57] <rochberg> I would like to bounce the x86-mario canary if possible. If it passes, then this smells like a "file a bug" thing and not a "close the tree" thing [18:58:59] <cmasone> gsam_: dave? Oh, chromium sheriffing [18:59:13] <cmasone> rochberg: I think cycling the canary takes many hours [18:59:46] <dlaurie> yeah I started one on x86-alex-canary after your change went in but it does seem to take a long time [19:01:28] <cmasone> I mean, I still think this is a file-a-bug thing [19:02:33] <gsam_> i'm ok w/ opening [19:02:52] * gsam_ still trying to get oriented [19:02:59] <cmasone> it might be cool to ping troopers at chromium dot org and ask for the /var/log/messages from the _host_ to see if maybe we can figure out what was hapopening during that time window [19:03:33] <rochberg> x86-mario canary -> rochberg [19:03:33] <rochberg> x86-alex canary -> cmasone thinks he's fixed [19:03:33] <rochberg> x86-zgb canary -> chrome segfault [19:03:33] <rochberg> x86-zgb-binary -> chrome segfault [19:03:34] <rochberg> x86-alex-binary -> chrome segfault [19:03:40] <rochberg> I think we should open [19:03:53] <gsam_> +1 [19:04:08] <dlaurie> yeah I agree [19:04:12] *** saintlou has joined #chromium-os [19:04:21] *** ChanServ sets mode: +v saintlou [19:05:49] <crosbot> tree became 'Tree is open. rochberg filing bug on x86-mario canary. cmasone has potential fix for x86-alex-binary. rest is sig11' [19:06:35] <crosbot> tree became 'Tree is open. rochberg filing bug on x86-mario canary. cmasone has potential fix for x86-alex-binary. rest is sig11 (classified as 17899)' [19:08:10] *** jshin has left #chromium-os [19:09:26] *** behdad has quit IRC [19:10:35] *** behdad has joined #chromium-os [19:11:16] *** behdad has quit IRC [19:13:06] <cros_> Anybody verified that chromium OS works using the Open GL ES backend on an ARM platform? [19:15:16] <davidjames> rochberg: I'm working on a fix for the 'UploadPrebuilts' flaky failure we saw earlier on alex-binary 809 [19:16:39] <vpalatin> cros_: yes we are using that every day [19:16:45] <vpalatin> (on tegra based platform) [19:21:36] <cros_> vpalatin: Is the graphics driver source available? I've been trying to bring up chromium OS on a new ARM SoC with my own open GL ES implementation and graphics driver but am running into what seems like some synchronization issue. [19:23:30] <vpalatin> cros_: no the nvidia stuff is closed source [19:24:05] <vpalatin> (I mean the openGLES shared libraries, the kernel part is available in the repository) [19:24:37] <Hexxeh> The most up to date versions aren't available, but you can grab nVidia's LDK from their site and get equivalent versions. [19:24:43] <Hexxeh> @ cros_ [19:25:07] <cros_> vpalatin: Do you have any idea if their driver uses XPutImage to move data from the GPU to output to the screen? Any idea what frame rate you get for UI composition on your tegra platform? [19:25:33] <rochberg> http://code.google.com/p/chromium-os/issues/detail?id=18968 filed on cryptohome [19:27:17] <vpalatin> cros_: I'm not sure which part you are calling the "driver" , but I don't think so. [19:28:09] <tlambert> @rochberg: that failure is the same SIGSEGV that happens in the Chrom itself when the MessagePump tries to tear things down at the same time it's trying to start a task. [19:28:42] <cros_> vpalatin: I am referring to libEGL. [19:28:46] <tlambert> It's basically a timing race on session shutdown. [19:29:20] <tlambert> you can see that it's didn't really fail where it appears: suite_Smoke/login_CryptohomeMounted PASS [19:29:24] <gsam_> are we seeing this only in qemu? [19:29:32] <tlambert> Generally, yes. [19:29:44] <cros_> vpalatin: Do you know how they move the GPU output to the screen? Are they using direct framebuffer rendering? [19:30:16] <tlambert> It requires timing. It appears to have been happening since some time last year, off and on. See also 178899 [19:30:23] <tlambert> 17899 [19:30:28] <rochberg> tlambert: where is the sig11 in the logs then? [19:31:21] <aaronp> cros_, no, EGL renders directly on the GPU and then uses a private protocol to pass the image off to the X server. [19:32:24] <Hexxeh> aaronp: Can you check whether there's a bug filed for this error for me please? Shader translator allowed/produced an invalid shader [19:32:29] <tlambert> open the link for stdio that you posted :) and then search for the string "chrome sig 11" [19:32:58] <rochberg> How did I miss that?? [19:32:59] <tlambert> Or the send "fail", if searching in chrom, and then go up about 20 lines [19:33:00] <rochberg> Thanks [19:33:36] <rochberg> Oh---wait----I'm looking at the second set of tests [19:33:57] <tlambert> The earliest formal report including enough info to get a back trace is 11407, from last January. [19:34:28] <aaronp> Hexxeh, I'm not finding one with that string in it. [19:34:31] <cros_> aaronp: What do you mean "private protocol"? [19:34:36] <tlambert> But davidjames provided info from modifying it so we got dumps. [19:34:47] <aaronp> I mean an NVIDIA custom X11 extension called "TDR". [19:35:04] <aaronp> It's akin to the NV-GLX extension in the desktop driver. [19:35:19] <Hexxeh> aaronp: Hmm, one of three errors I can find in all the logs, nothing else seems to explain the sluggishness. Only other thing is a square block around the cursor that appears sometimes. [19:35:33] <aaronp> Are you using an ARGB cursor? [19:35:51] <cros_> aaronp: Any reason XPutImage wouldn't work? I am witnessing some sort of timing / synchronization issue I suspect. [19:36:02] <Hexxeh> aaronp: I'm using whatever the defaults are in terms of cursors [19:36:02] <aaronp> ARGB hardware cursors are disabled by default right now because the kernel support for them is not great, and software cursors are known to cause artifacts sometimes. [19:36:16] <aaronp> cros_, didn't you say you were using the fbdev driver? [19:36:39] <Hexxeh> iirc that was commented out in the xorg config by default? so it'd be software cursors I guess yeah [19:37:11] <aaronp> You can try enabling hardware cursors using the xorg.conf option, but your best bet is to just stick with 2-color cursors for now. [19:38:28] <Hexxeh> aaronp: The biggest problem is the fact it runs like molasses, cursor is a minor issue [19:39:22] <cros_> aaronp: Based on what I see in the X server log, yes, but I could be misunderstanding you since I don't know much about X. Is there a reason using XPutImage wouldn't work and/or why nvidia uses 'TDR'? [19:39:55] <aaronp> Did you ever determine whether your clocks were at the correct speeds? [19:40:12] <Hexxeh> they were scaling up, that didn't appear to be the issue, and setting it to performance didn't help [19:40:38] <ellyjones> anyone here know about boot paths for qemu and alex? I am curious how they differ [19:40:55] <aaronp> cros_, using XPutImage would require the EGL driver to copy the image data from GPU-accessible memory to system memory, then pack that into the X data stream to send to the server, where the server would unpack it into another chunk of system memory, then transfer it back to the GPU. [19:40:57] <cmasone> ellyjones: we boot inside qemu? [19:41:03] <ellyjones> yeah, for tests [19:41:05] *** dnicoara has joined #chromium-os [19:41:06] *** ChanServ sets mode: +v dnicoara [19:41:12] <ellyjones> but the boot path is apparently difference since one of my recent changes broke that [19:41:15] <aaronp> It's three extra copies that are not necessary. [19:41:19] <ellyjones> and now I'm trying to figure out wtf [19:41:22] <aaronp> You need to be using the driver called "tegra". [19:41:32] <aaronp> Look for "TEGRA(0)" in /var/log/Xorg.0.log [19:41:33] <crosbot> tree became 'Tree is closed (Automatic: "update" on "stumpy-binary")' [19:41:58] <ellyjones> error: The requested URL returned error: 403 while accessing http://gerrit.chromium.org/gerrit/p/chromiumos/chromite/info/refs [19:42:01] <ellyjones> uhoh [19:42:02] <cros_> aaronp: I'm not using tegra, I'm bringing up an unsupported SoC. [19:42:12] <cmasone> ellyjones: troopers!! [19:42:18] <ellyjones> yeah, them! [19:42:22] <ellyjones> crosbot: troopers? [19:42:22] <crosbot> ellyjones: troopers: djmm, scottz, bradnelson, maruel (EST), kliegs (EST) [19:42:22] <cmasone> crosbot: troopers? [19:42:23] <tlambert> ellyjones: where in the boot path did it break? There are some significant differences. [19:42:23] <crosbot> cmasone: troopers: djmm, scottz, bradnelson, maruel (EST), kliegs (EST) [19:42:23] <Hexxeh> what are troopers? :P [19:42:35] <ellyjones> tlambert: I have no idea :\ I got no debug output [19:42:39] <Hexxeh> how do they differ to sheriffs? [19:42:45] <ellyjones> Hexxeh: mythical beings with the credentials and knowhow to fix the buildbots [19:42:46] <gsam_> why is stumpy closing the tree [19:42:51] <aaronp> cros_, oh, sorry, I was confused I guess. In that case, good luck. [19:42:59] <Hexxeh> ellyjones: ahh, okay [19:43:18] <ellyjones> cros_: I keep tab-completing your nick instead of crosbot :P [19:43:34] <tlambert> ellyjones: then how do you know it is broken? :p [19:43:45] <kliegs> ellyjones: I thought you were supposed to be on that list :) [19:43:47] <ellyjones> tlambert: because it never boots [19:43:55] <cros_> aaronp: I didn't get the last part of your previous statement. Why would you have to transfer back to the GPU? [19:43:56] <ellyjones> kliegs: no, because scottz hasn't actually done the thing [19:44:44] <daveparker> I have an OKR to be more social. Hello everyone :) [19:44:54] <ellyjones> hi daveparker [19:45:00] <ellyjones> do _you_ know about qemu boot? :P [19:45:10] <ellyjones> tlambert: I see "Booting from hard disk...", then nothing [19:45:18] <aaronp> cros_, is your EGL implementation rendering in software using the CPU, or using a GPU? [19:45:28] <ellyjones> I can't figure out how to convolve my qemu arguments or kernel commandline to get console output [19:45:34] <Hexxeh> aaronp: would fail on line 53 here manifest itself in the form of lack of graphical acceleration? http://src.chromium.org/svn/trunk/src/ui/gfx/gl/gl_surface_egl.cc [19:45:56] <tlambert> ellyjones: There are some significant differences in the xorg.conf that impact booting. Currently, the closed source synaptics driver generally takes about an extra couple of seconds on the closed source. If you try and boot a normal image, it's going to fail in X initialization because it's going to be unable to find mandatory input devices. [19:46:22] <ellyjones> tlambert: yeah, I know; I used image_to_vm.sh to generate a qemu image [19:46:25] <ellyjones> then tried to boot that in qemu [19:46:39] <kliegs> crosbot: sheriffs? [19:46:39] <crosbot> kliegs: sheriffs: dlaurie, sleffler, zork, rochberg [19:46:39] <ellyjones> in fact, if I build an image without my change, and use image_to_vm, it boots fine in qemu [19:46:42] <tlambert> So the X config is different to change input devices to not try to use that driver. [19:46:49] <ellyjones> yeah [19:46:57] <gsam_> Cloning into chromite... [19:46:57] <gsam_> error: The requested URL returned error: 403 while accessing http://gerrit.chromium.org/gerrit/p/chromiumos/chromite/info/refs [19:46:57] <gsam_> fatal: HTTP request failed [19:46:57] <gsam_> cd: 1: can't cd to chromite [19:46:57] <gsam_> fatal: Not a git repository (or any of the parent directories): .git [19:47:00] *** daveparker has quit IRC [19:47:02] <ellyjones> but if I build an image with my change, it boots on real hardware but not on qemu [19:47:03] <kliegs> dlaurie, rochberg: have you just tried bouncing stumpy to see if its transient? [19:47:15] <ellyjones> a transient 403? oO [19:47:18] <cmasone> gsam_: yah, we saw that. Ping a trooper? [19:47:26] <ellyjones> kliegs is a trooper [19:47:37] <tlambert> ellyjones: CL in gerrit I can look at or anything? [19:47:37] <aaronp> Hexxeh, it looks like g_native_display would just be uninitialized in that case. If you can't get a connection to the X server, it should just fail to initialize completely. [19:47:41] <gsam_> kliegs, it's been failing for a while [19:47:43] <cros_> aaronp: What exactly do you mean by rendering in EGL? For our GPU from the render target to the framebuffer the rendering will be done by the CPU. [19:47:50] <aaronp> er, NULL, not uninitialized. [19:47:51] <gsam_> kliegs, can we reboot? [19:47:51] <ellyjones> tlambert: yeah - http://gerrit.chromium.org/gerrit/#change,3970 [19:47:53] <kliegs> gsam_: not with that error. and i just ran the command from my machine fine [19:47:58] <Hexxeh> aaronp: I've seen that error in the logs before now, but it hasn't caused things not to work at all [19:48:16] <aaronp> cros_, okay, in that case, XPutImage (or rather, XShmPutImage) would probably be your best bet. [19:48:27] <Hexxeh> aaronp: I guess the problem is I'm debugging blind, I can't see what's going wrong to try and fix it [19:48:30] <aaronp> For us, it would be horrible because of the extra copies which is why we do it differently. [19:48:32] <kliegs> gsam_: I can bounce it sure. [19:48:37] <ellyjones> gsam_: kliegs: if I fetch that URL myself, I also get a 403 [19:48:43] <ellyjones> I really doubt bouncing the bot will help [19:48:58] <kliegs> /usr/bin/git clone http://gerrit.chromium.org/gerrit/p/chromiumos/chromite gives you a 403? [19:49:01] <kliegs> it just worked for me [19:49:13] <kliegs> if so it sounds like git mirroring/infrastructure issue [19:49:18] <ellyjones> hrm, git clone works, but that URL is a 403 [19:49:25] <ellyjones> that is odd [19:49:43] <gsam_> is x86-zgb-binary fail the old failure? [19:49:55] <gsam_> nm [19:50:21] <kliegs> ellyjones: i just put the link into a browser and it loaded gerrit for me [19:50:22] *** jrbarnette has quit IRC [19:50:29] <ellyjones> I put it into a browser and got a 403 [19:50:31] <ellyjones> weird! [19:50:47] <ellyjones> if you cd into that directory on the buildbot and 'git remote update' what happens? [19:50:49] <gsam_> fails for me [19:51:22] <Hexxeh> aaronp: Got any more ideas for things to check? I'm pretty much at a dead end since I don't know all the places to check for Tegra [19:51:44] <Hexxeh> aaronp: I remember having odd failures when /tmp wasn't mounted with exec on x86 with the nvidia binary drivers, does the same apply for the Tegra drivers? [19:51:46] <cros_> aaronp: When you refer to TDR are you referring to Time-out Detection and Recovery ? [19:51:53] <kliegs> ellyjones: i'm suspecting its a git server issue then, not an issue with the bot [19:52:06] <Hexxeh> cros_: I'd hazard a guess it stands for something like Tegra DRiver? [19:52:19] <ellyjones> yeah, ok [19:52:20] <Hexxeh> Probably wrong, mind... :P [19:52:25] <ellyjones> maybe bounce it then :P [19:52:26] <kliegs> gsam_: unfortunately I don't have the access/ability to debug git backend issues - can you please file a bug against chrome-infrastructure-team and get their attention for that? [19:52:56] <tlambert> ellyjones: we are using devtmpfs; those majors and minors are going to move around on you because (a) the VM has different "hardware" and (b) they depend on unspecified device driver load order and (c) they depend on unspecified udev event order creating devices like /dev/serio_raw0 for the closed source driver. Am I missing something? My hunch would be the udev config for the Synaptics touchpad. [19:53:01] <dlaurie> hm I tried to bounce stumpy but it isn't starting [19:53:11] <cmasone> gsam_ should probably start by pinging troopers at chromium dot org like it says in the sheriffing FAQ [19:53:26] <gsam_> roger [19:53:39] <aaronp> Hexxeh, I don't think /tmp should matter, but I don't know for sure. [19:53:39] <kliegs> cmasone, gsam_: is that needed if it looks like a git backend issue? [19:53:43] <aaronp> You're not using GLX, are you? [19:53:47] <kliegs> and if I'm already starting to look at the bot? [19:53:49] <Hexxeh> aaronp: GLX is disabled [19:53:52] <aaronp> No, you said EGL already. [19:53:57] <Hexxeh> yeah [19:54:03] <cmasone> kliegs: yes [19:54:08] <aaronp> And you're sure you're getting the NVIDIA EGL and not some Mesa one, right? [19:54:22] <Hexxeh> it just seems odd that i'm not getting any errors, and yet things aren't working, you'd expect that something would be shouting things are wrong since they are [19:54:29] <Hexxeh> aaronp: using libEGL from the LDK [19:54:47] <Hexxeh> i'm literally just doing a tegra2_dev-board build then using the LDK files dropped on top [19:55:00] <cros_> aaronp: I'm not sure if you're talking to Hexxeh or me. [19:55:09] <aaronp> I'm talking to Hexxeh. [19:55:12] <Hexxeh> going to submit patches to use alternatives to the private git if you're outside of google once it works [19:55:15] <kliegs> This step from the doc: "Sometimes, a tree closure is due to some kind of infrastructure problem. In that case, quickly escalate to a Trooper."? [19:55:18] <aaronp> All the /dev/nv* device files are accessible by your user? [19:55:22] <Hexxeh> yep [19:55:30] <aaronp> Hmm. Yeah, I don't know then, sorry. [19:55:36] <Hexxeh> added hacks in ui.conf for that, /dev/nv* and /dev/tegra* are all 0666 [19:55:42] <aaronp> cros_, TDR stands for Tegra Direct Rendering. [19:56:05] <kliegs> gsam_: FYI - stumpy bot last rebooted 13 minutes ago. most bots reboot on cycle completion so this looks normal [19:56:37] <Hexxeh> udev rules didn't seem to work for whatever reason [19:57:01] <gsam_> mail sent [19:58:45] <cros_> aaronp: Thanks, I don't find much on google regarding TDR. Anywhere I can get more information on that? [19:59:14] <Hexxeh> It's a private thing, so probably not [19:59:58] <aaronp> cros_, yeah, what Hexxeh said. Also it wouldn't be useful for you if you're not using Tegra. [20:00:21] <Hexxeh> aaronp: I do wonder if the error is somewhere in my kernel port, since obviously the official 2.6.38-chromeos doens't support the TF101 [20:01:03] <Hexxeh> That said, I'll be sending patches upstream for it. It'd be nice to get some support for non-official Tegra 2 devices running, I heard one guy getting 19 hours with Ubuntu on a Transformer the other day, imagine what you'd get on CrOS....! [20:01:52] <cros_> aaronp: If my GPU performance is very slow, less than 10 frames per second, will the chromium window manager still function or do we need to adjust that 16ms timeout? [20:01:53] <aaronp> Possible. There are a pile of clock trees that affect various parts of the chip besides the CPU, and if any of those are not getting raised it could also impact performance. [20:02:03] <aaronp> cros_, no idea. [20:02:28] <Hexxeh> aaronp: Got any specifics I could check? [20:02:56] <Hexxeh> U-Boot works great on the TF101, too [20:04:20] <gsam_> yay [20:04:35] <aaronp> Hexxeh, cat /sys/kernel/debug/clock/clock_tree [20:05:03] <Hexxeh> aaronp: Will just finish up this battery driver then load her up and take a look, thanks :) [20:05:18] *** sergiu has joined #chromium-os [20:05:19] *** sergiu has joined #chromium-os [20:06:02] <gsam_> Hexxeh, you still working on mbair support? [20:06:11] <Hexxeh> gsam_: It's pretty much done [20:06:24] <gsam_> Hexxeh, are we going to fold it into our tree? [20:06:35] <gsam_> (modulo binary bits) [20:07:12] <Hexxeh> gsam_: Probably not, since I doubt the nVidia binary driver is welcome (I think I asked at some point), and it will ONLY boot from a MBR image, so the 12 partition GPT setup is a no-go [20:07:28] <Hexxeh> The upshot though is that it does work VERY well on an Air [20:07:31] <gsam_> Hexxeh, ok, thanks for your work regardless [20:07:49] <Hexxeh> I've kicked off OSX and have been running it solely on my Air since I finished it, works great [20:08:36] <Hexxeh> gsam_: Doing GSoC mostly at the moment (which I really should get back to before my mentor lynches me), but I'll take a look at trying to submit patches that allow it in a couple weeks once that's finished [20:09:19] <Hexxeh> I could add an MBR image as an option, but of course it'd break AU [20:10:09] <Hexxeh> Definitely interested in submitting patches to make the Tegra2 stuff more open, though [20:11:11] <powderluv> Hexxeh: I think the new nouveau driver should work if you boot from EFI [20:11:25] <powderluv> however it may require a grub efi update [20:11:28] <Hexxeh> powderluv: that was one suggestion, however there are more problems with EFI boot on the Air [20:11:43] <Hexxeh> not even framebuffer works on the Air under EFI [20:11:55] <powderluv> huh really ? [20:12:01] <Hexxeh> there were kernel patches that fixed it, but they weren't accepted upstream [20:12:03] <powderluv> I got it to work [20:12:09] <powderluv> ahh yeah [20:12:23] <powderluv> i had that one patch [20:12:28] <powderluv> locally [20:15:10] <gsam_> i see no answer from troopers [20:15:20] <gsam_> any suggestions on stumpy and tree closed? [20:15:30] <cmasone> gsam_: that might be a worthy thing to put in the postmortem :-) [20:15:45] <cmasone> gsam_: do we have reason to believe the 403 won't just happen again? [20:15:50] <gsam_> open an issue? [20:15:52] <Hexxeh> aaronp: Anything specific to check in the clocktree? [20:16:14] <gsam_> cmasone, sorry but i have no answer [20:16:44] <cmasone> gsam_: def open an issue [20:17:02] <gsam_> works for me now [20:17:02] *** powderluv has quit IRC [20:17:08] <gsam_> can open url in browser [20:17:10] *** powderluv has joined #chromium-os [20:17:11] *** ChanServ sets mode: +v powderluv [20:18:00] *** kerz has joined #chromium-os [20:18:01] *** ChanServ sets mode: +v kerz [20:18:27] <dlaurie> I tried to kick stumpy but it never started a build [20:18:27] <gsam_> kicked stumpy [20:18:32] <gsam_> oh [20:18:36] <dlaurie> maybe you'll have more luck :) [20:18:40] <gsam_> we'll see [20:18:48] <cmasone> it probably shares a bot with something else. [20:19:49] <cmasone> gsam_: maybe also ping scottz directly with that troopers email? That should have gone to him, but god knows if it did [20:22:25] <crosbot> tree became 'Tree is closed (Automatic: "update" on "stumpy-binary") (git now reachable but but won't restart)' [20:23:11] <ojn> First irc login in 3 weeks and my tab doesn't go blue. Excellent. :) [20:23:27] *** JasonO has joined #chromium-os [20:23:28] *** JasonO has joined #chromium-os [20:25:37] <davidjames> dlaurie: I've seen that issue too where a bot just refuses to start [20:26:46] <crosbot> tree became 'Tree is closed (Automatic: "update" on "stumpy-binary") (18974)' [20:26:49] <davidjames> dlaurie: Restarting the buildbot on the machine in question gets it to startup pretty quick though -- it's started now :) [20:26:50] <gsam_> if someone can get the bot moving i suppose we can re-open the tree [20:27:09] <davidjames> gsam_: Done [20:27:22] <dlaurie> ok it failed pretty quickly last time so hopefully we can tell it is ok soon [20:27:24] <gsam_> all i see is red on the waterfall [20:28:04] <davidjames> gsam_: Yeah that's mostly cause the test suite fails consistently [20:28:38] <davidjames> *we consistently see failures on at least one bot at any given time :) [20:29:05] <gsam_> davidjames, did you open the tree? [20:29:21] <davidjames> gsam_: Nope [20:29:38] <gsam_> ha love the msgs in the stumpy tab about everyone pushing the button [20:30:18] <davidjames> gsam_: Yes at least it was smart enough to remember all the times the button was pushed :) [20:30:42] *** yoshiki_ has quit IRC [20:30:59] <kliegs> gsam_: stumpy bot is shared with others i think [20:31:12] <gsam_> thanks [20:31:33] <kliegs> oh. it did start, just hung? [20:31:45] <gsam_> kliegs, davidjames kicked it [20:31:50] <kliegs> gsam_: ahh. ok [20:31:51] <dlaurie> it is working, just slowlyh [20:32:03] <dlaurie> svn: Can't connect to host 'svn.chromium.org': Connection timed out [20:32:04] <dlaurie> or not... [20:32:04] <kliegs> gsam_: as for the git error, that needs to go to the infrastructure team, not the troopers [20:32:15] <gsam_> oops [20:32:21] <kliegs> if people outside the bot were seeing the same thing (ellyjones replicated) its not a bot issue [20:32:48] <gsam_> i feel like i'm in a maze of twisty passages all alike :) [20:33:17] <dlaurie> I am being nibbled on by a grue [20:33:45] *** Styx has quit IRC [20:34:42] <gsam_> kliegs, how do i direct http://code.google.com/p/chromium-os/issues/detail?id=18974 to the right people? or is fwd'ing the email enough? [20:35:53] <kliegs> gsam_: add chrome-infrastructure-team at google dot com to the cc list [20:36:44] <kliegs> if you need quicker responses from them, you can always escalate to those on the list through IRC, chat, phone or sneakernet [20:36:46] <davidjames> gsam_: cmp is the trooper-on-duty today [20:37:05] <gsam_> svn: Can't connect to host 'svn.chromium.org': Connection timed out [20:37:23] * gsam_ always a step behind dlaurie [20:39:19] *** cros_ has quit IRC [20:40:53] *** satorux_ has joined #chromium-os [20:40:53] *** ChanServ sets mode: +v satorux_ [20:45:30] *** jrbarnette has joined #chromium-os [20:45:30] *** ChanServ sets mode: +v jrbarnette [20:46:48] <gsam_> should we reopen? [20:48:29] <davidjames> gsam_: If you do, note that arm-generic PFQ and tegra2 PFQ are going to close the tree soon because they just failed in 'sync buildbot slave files' stage [20:48:52] <davidjames> svn: Can't connect to host 'svn.chromium.org': Connection timed out [20:49:08] <gsam_> yes, that was added to the issue [20:49:20] <gsam_> oh [20:49:29] <davidjames> gsam_: Cool, yeah, just get ready to reopen it when it autocloses :) [20:50:18] <crosbot> tree became 'Tree is closed (Automatic: "update" on "stumpy-binary") (18974, now failing with Can't connect to host 'svn.chromium.org')' [20:53:58] <davidjames> gsam_: Apparently SVN is down, this is causing even more trouble for Chromium folks [20:54:16] <davidjames> gsam_: So expect all builds to start failing soon if they don't fix SVN [20:54:26] <davidjames> (Well, all builds that build Chrome from source) [20:55:00] <davidjames> gsam_: cmp is working on it right now [20:56:52] <kliegs> davidjames: I think all builds check out buildbots from source as their start. so I think they'll all hit it [20:57:05] <gsam_> stumpy just went purple! [20:57:49] *** msb__ has quit IRC [20:57:51] <gsam_> i think today is karmic payback for my last few sheriff assignments [20:59:42] *** patcito has joined #chromium-os [21:04:57] *** cros_ has joined #chromium-os [21:05:15] <davidjames> gsam_: Oh, yeah, scottz is fixing the machine right now [21:05:29] <gsam_> davidjames, is it just one machine? [21:05:48] <gsam_> if so should we re-open the tree? [21:06:08] <davidjames> gsam_: Yeah if stumpy is the only issue, no need to close the tree as it should be back up soon [21:06:42] <gsam_> davidjames, what about the arm pfq failures? [21:06:53] <gsam_> looks like they cycled green [21:07:22] <gsam_> ok, gonna reopen [21:07:43] <davidjames> gsam_: Yeah looks fine. Note that Chromium tree is still closed due to svn issues, but if we're not seeing anything, maybe the svn issues aren't so bad :) [21:08:11] <crosbot> tree became 'Tree is open (stumpy failure being fixed by scottz)' [21:10:58] <davidjames> gsam_: I wonder if we should whitelist the Chrome crashes while they're being worked on [21:10:58] <Hexxeh> If I've added a kernel driver for my battery, but ChromeOS still isn't displaying the status, what package needs fixing? [21:11:30] <gsam_> davidjames, i see a change from derat in the q [21:11:33] <davidjames> gsam_: autotest actually has the ability to ignore certain crashes, so we would have the capability to ignore the crashes if we are tired of tree closing again and again while they're being worked on [21:11:56] <gsam_> davidjames, seems like much of this morning has been infrastructure issues [21:12:57] <davidjames> gsam_: well, stumpy + the svn issues, yup [21:14:49] <dlaurie> Hexxeh: the UI gets its info from devkit-power I believe [21:16:17] <davidjames> Hexxeh: You might want to look at power_supply.cc in power_manager [21:16:57] <Hexxeh> running power-supply-info shows that it's trying to check the presence of the dock battery, rather than the internal battery [21:18:14] *** eldar has joined #chromium-os [21:22:11] *** saintlou has quit IRC [21:22:30] *** saintlou has joined #chromium-os [21:34:10] *** D|sT has quit IRC [21:39:56] *** Kernel-Panic has quit IRC [21:41:24] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "tegra2_arthur-binary" from 25978083d957f6cb0a775a82f9011ce0e6e0ad84: estade at chromium dot org <estade at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>, maruel at chromium dot org <maruel at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>)' [21:42:40] <gsam_> repo sync failed [21:43:14] <cmasone> gsam_: trooper-de-dooper? did it hang for 2.5 hours? [21:43:59] <gsam_> unlikely [21:44:41] <cmasone> gsam_: huh. We've seen that happen fairly often; there's a bug for it [21:45:33] *** D|sT has joined #chromium-os [21:45:35] <gsam_> why does it continue after; or am i mis-reading the log [21:45:57] <gsam_> nm [21:46:03] <dlaurie> yeah that was the 10am build, I am confused [21:46:26] *** cros__ has joined #chromium-os [21:46:38] <cmasone> it looks like the 10AM build hung trying to sync for 2+ hours (a thing we have a bug for) and then a new build started right after [21:46:48] *** cros_ has quit IRC [21:47:11] *** Kernel-Panic has joined #chromium-os [21:47:19] <sergiu> I had a sync problem yesterday evening as well [21:47:23] <cmasone> at 12:11 or so. And that build succeeded in (relatively) short order. [21:49:09] * gsam_ goes to forage lunch [21:51:40] <Hexxeh> How does power_manager decide which battery to use under /sys/class/power_supply? [21:51:44] *** DoDzy has quit IRC [21:52:06] <Hexxeh> It thinks it has no battery because the present property isn't there on the battery it's trying to look at [21:52:20] *** dtu has joined #chromium-os [21:52:20] *** ChanServ sets mode: +v dtu [21:54:55] <dlaurie> the tegra2_arthur cycled green, time to re-open? [21:55:35] *** Kernel-Panic has quit IRC [21:57:47] <dlaurie> hm alex-binary failed test too, looking at it [21:59:18] <dlaurie> looks like 18788 [22:00:15] <crosbot> tree became 'Tree is open (tegra2_arthur-binary cycled green, x86-alex-binary test failure was 18788)' [22:03:39] <cmasone> dlaurie: I can get to it in a little bit, but if you want to send me a patch that comments out that unit test for now, I will be happy to review it [22:03:52] <dlaurie> ok I will do that [22:05:39] *** saintlou_ has joined #chromium-os [22:07:53] *** 92AADTP01 has quit IRC [22:07:55] *** 31NAAURFA has quit IRC [22:08:11] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot" on "stumpy-binary" from cb6dbc5953beafe4801716bf8e87e3db771cc134: jochen at chromium dot org <jochen at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>, senorblanco at chromium dot org <senorblanco at chromium dot org@2bbb7eff-a529-9590-31e7-b0007b416f81>, serya at chromium dot org <serya at chromium dot org@0039d316-1c4b-4281-b951-d872f2087c98>)' [22:08:16] *** rbyers_ has joined #chromium-os [22:08:19] *** saintlou has quit IRC [22:10:04] <dlaurie> looks like the same chrome sig112 [22:10:07] <dlaurie> er, sig11 [22:11:00] <crosbot> tree became 'Tree is open (stumpy-binary was 17899)' [22:13:08] <dlaurie> is there an easier way to disable other than #ifdef the whole test out? [22:14:38] <cmasone> dlaurie: you can also prefix the test name with DISABLED_ [22:15:07] <dlaurie> ah ok [22:15:59] *** 18WABB2FY has joined #chromium-os [22:15:59] *** 92AADTVU2 has joined #chromium-os [22:21:12] <dlaurie> ugh I can't submit it, can you submit when you approve? [22:21:33] *** msb__ has joined #chromium-os [22:21:33] *** ChanServ sets mode: +v msb__ [22:21:40] <cmasone> dlaurie: dun! [22:22:11] <ellyjones> wfrichar: fyi, I am about to land http://gerrit.chromium.org/gerrit/#change,5668 hopefully :) [22:33:52] <adlr> gr. can't upload to gerrit. dang remote end keeps hanging up on me [22:36:54] *** MalelDraconis has joined #chromium-os [22:38:27] *** MalelDraconis has left #chromium-os [22:45:49] <adlr> can anyone else upload, or is my own machine screwed up? [22:46:03] <ellyjones> I uploaded a few minutes ago successfully [22:46:04] <cmasone> adlr: I just uploaded, though it was not a new patch [22:46:17] <adlr> hm, okay [22:46:20] <adlr> thanks [22:46:26] <ellyjones> got ssh keys, adlr? [22:47:14] *** simon____ has quit IRC [22:47:14] <msb__> adlr: just uploaded a new patch [22:47:29] <crosbot> tree became 'Tree is closed (Automatic: "cbuildbot_master" on "TOT Pre-Flight Queue" from 73b070f6bea45a7e75035ff6bbbe0bd73d1e74f3: arthurhsu at google dot com <arthurhsu at google dot com@672e30a5-4c29-85ac-ac6d-611c735e0a51>)' [22:47:42] <adlr> yeah, ssh keys was it. thanks guys [22:48:04] <ellyjones> I always forget to ssh-add after weekends [22:50:50] <dlaurie> looks like the same chrome crash, I don't like it but should we just reopen again? [22:52:05] <crosbot> tree became 'Tree is open (chrome sig11)' [22:57:34] *** JasonO has quit IRC [23:05:37] *** rharrison_chrome has quit IRC [23:06:18] *** rharrison_chrome has joined #chromium-os [23:06:19] *** ChanServ sets mode: +v rharrison_chrome [23:06:34] *** rbyers_ has quit IRC [23:07:27] *** rharrison_chrome has quit IRC [23:11:56] *** Kernel-Panic has joined #chromium-os [23:12:36] *** Kernel-Panic has joined #chromium-os [23:13:17] *** Kernel-Panic has joined #chromium-os [23:13:57] *** Kernel-Panic has joined #chromium-os [23:14:34] *** Kernel-Panic has joined #chromium-os [23:15:14] *** Kernel-Panic has joined #chromium-os [23:15:53] *** Kernel-Panic has joined #chromium-os [23:16:33] *** Kernel-Panic has joined #chromium-os [23:24:56] *** rginda_home has joined #chromium-os [23:28:40] *** achuith has quit IRC [23:28:48] *** achuith has joined #chromium-os [23:28:49] *** ChanServ sets mode: +v achuith [23:29:38] *** achuith has quit IRC [23:30:12] *** achuith has joined #chromium-os [23:30:13] *** ChanServ sets mode: +v achuith [23:30:44] *** rush2end_ has joined #chromium-os [23:32:27] *** rush2end has quit IRC [23:32:37] *** rush2end_ is now known as rush2end [23:38:46] *** Styx has joined #chromium-os [23:44:26] *** saintlou_ has quit IRC [23:49:51] *** petermayo has quit IRC [23:53:23] <gsam_> what just happened to the pfq? [23:53:30] <gsam_> nm