Switch to DuckDuckGo Search
   November 16, 2019  
< | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | >

Toggle Join/Part | bottom
[00:00:18] <rmustacc> Right now the OS is programmed to die on them. Why enabling link bandwidth monitoring on the bridge would trigger this is a bit more of a mystery to me.
[00:00:23] <rmustacc> But hardware works in mysterious ways.
[00:00:40] <rmustacc> So the question I have is should we be considering them fatal when the PCIe spec says they're advisory.
[00:00:55] <rmustacc> I'll do a little spelunking on this and may have something which suggests that we shouldn't treat it as a fatal error.
[00:01:06] <rmustacc> I'm kind of curious if we do that, how well those systems work in both case.
[00:01:09] <rmustacc> *cases
[00:09:47] <rmustacc> despair86: When you did the test of setting pcieb_die to zero, did all the devices seem to still work OK?
[00:10:38] <despair86> yes
[00:10:56] <rmustacc> OK, that's good to know. Just helps with some of the thoughts I have here.
[00:15:03] <despair86> yeah i'm even doing a full kernel build atm, forgot to cap dmake jobs so some stuttering going on too
[00:20:12] *** gh34 <gh34!~textual@cpe-184-58-181-106.wi.res.rr.com> has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
[00:21:33] *** Kurlon <Kurlon!~Kurlon@cpe-67-253-141-249.rochester.res.rr.com> has joined #illumos
[00:49:31] <gitomat> [illumos-gate] 11968 APIC calibration should explicitly initialise the PIT -- Joshua M. Clulow <josh at sysmgr dot org>
[00:50:50] *** mnrmnaugh <mnrmnaugh!~mnrmnaugh@unaffiliated/mnrmnaugh> has quit IRC (Ping timeout: 246 seconds)
[00:57:17] *** andy_js <andy_js!~andy@94.11.180.149> has quit IRC (Quit: andy_js)
[01:03:01] *** mnrmnaugh <mnrmnaugh!~mnrmnaugh@unaffiliated/mnrmnaugh> has joined #illumos
[01:04:56] <rmustacc> zsj: If you're having the same problems with the chipset and PCIe errors, I've posted an update to 11953 about it with a hopeful initial fix.
[01:05:53] <zsj> thanks! yeah, I'm János on that report
[01:08:54] <rmustacc> Ah, gotcha. OK.
[01:09:32] *** CaptainTobin <CaptainTobin!~tobin@c-68-38-10-41.hsd1.in.comcast.net> has quit IRC (Read error: Connection reset by peer)
[01:15:14] *** jemersha- <jemersha-!~jemershaw@c-68-83-252-28.hsd1.pa.comcast.net> has joined #illumos
[01:17:57] *** jemersha| <jemersha|!~jemershaw@172.58.227.63> has joined #illumos
[01:18:21] *** jemershaw <jemershaw!~jemershaw@c-68-83-252-28.hsd1.pa.comcast.net> has quit IRC (Ping timeout: 268 seconds)
[01:20:35] <zsj> so I should follow https://illumos.org/books/dev/workflow.html and apply the patch after cloning illumos-gate, I guess? do I do this on the un-updated openindiana 2019.04 I have working, or do pkg update and chroot into the new boot environment that won't boot until this patch?
[01:21:16] *** jemersha- <jemersha-!~jemershaw@c-68-83-252-28.hsd1.pa.comcast.net> has quit IRC (Ping timeout: 240 seconds)
[01:21:20] <rmustacc> zsj: You normally use OI?
[01:21:44] <zsj> I installed OI for the first time this week. I normally use gentoo
[01:21:48] <rmustacc> Gotcha.
[01:22:11] <rmustacc> Let's see. I'm not sure if someone here is around who could build it for you.
[01:22:26] *** jemersha| <jemersha|!~jemershaw@172.58.227.63> has quit IRC (Ping timeout: 240 seconds)
[01:22:46] <rmustacc> Otherwise, I think if you basically follow https://illumos.org/docs/developers/build/, but apply that patch (and I can push that somewhere so you don't have to manually apply it).
[01:22:56] <rmustacc> And then use onu to create a boot environment with it and boot and give it a shot.
[01:23:20] <rmustacc> The good news about the new BE is that if it doesn't work, it should be easy to fail back.
[01:23:24] <rmustacc> Erm, fall back.
[01:23:51] <zsj> yeah, I did it a couple of times now :)
[01:24:02] <rmustacc> OK.
[01:24:39] *** jemershaw <jemershaw!~jemershaw@c-68-83-252-28.hsd1.pa.comcast.net> has joined #illumos
[01:26:55] <despair86> if anyone needs the patched pcieb driver i can serve that up in a bit...thank god sun moved to SVRr4 and dynamically loaded kernels :-)
[01:28:07] <zsj> I might need to reinstall anyway, the 2019.04 release doesn't set up the efi partition properly, and idk if I correct it manually if it'll keep it in sync after. right now I'm booting it with 'search -s -l rpool, chainloader /ROOT/openindiana/ at /boot/loader64 dot efi' added to my regular grub2 manually
[01:28:51] <zsj> I could use that I think
[01:28:52] <rmustacc> I'm honestly not sure there, sorry.
[02:20:00] *** ed209 <ed209!~ed209@165.225.128.67> has quit IRC (Remote host closed the connection)
[02:20:06] *** ed209 <ed209!~ed209@165.225.128.67> has joined #illumos
[02:26:20] *** Yogurt <Yogurt!~Yogurt@104-7-67-228.lightspeed.sntcca.sbcglobal.net> has quit IRC (Remote host closed the connection)
[02:30:06] *** CaptainTobin <CaptainTobin!~tobin@c-68-38-10-41.hsd1.in.comcast.net> has joined #illumos
[02:31:40] *** insomnia <insomnia!~insomnia@shadowcat/actuallyamemberof/lollipopguild.insomnia> has quit IRC (Ping timeout: 265 seconds)
[02:47:30] <despair86> ok, https://snowlight.net/files/bug/pcieb.tar.xz for anyone that can't build illumos-gate immediately
[02:52:52] *** despair86 <despair86!~despair@2605:6000:1515:45b:b62e:99ff:fea1:3bad> has quit IRC (Quit: brb installing patches)
[02:53:48] *** Qatz <Qatz!~db@2601:187:8400:5::d9d> has quit IRC (Read error: Connection reset by peer)
[02:53:58] *** Qatz <Qatz!~db@2601:187:8400:5::42d> has joined #illumos
[03:07:55] *** insomnia <insomnia!~insomnia@shadowcat/actuallyamemberof/lollipopguild.insomnia> has joined #illumos
[03:27:13] *** despair86 <despair86!~despair@2605:6000:1515:45b:b62e:99ff:fea1:3bad> has joined #illumos
[03:27:55] <despair86> just updated the tar with both 'misc/pcie' _and_ 'drv/pcieb'
[03:31:21] *** _despair86 <_despair86!~despair@2605:6000:1515:45b:b62e:99ff:fea1:3bad> has joined #illumos
[03:33:57] *** despair86 <despair86!~despair@2605:6000:1515:45b:b62e:99ff:fea1:3bad> has quit IRC (Ping timeout: 250 seconds)
[03:36:43] *** _despair86 <_despair86!~despair@2605:6000:1515:45b:b62e:99ff:fea1:3bad> has quit IRC (Quit: brb installing patches)
[03:37:03] *** despair86 <despair86!~despair@2605:6000:1515:45b:b62e:99ff:fea1:3bad> has joined #illumos
[03:54:41] *** _despair86 <_despair86!~despair@2600:1700:8f61:163f:36e8:94ff:fede:7ce2> has joined #illumos
[03:57:00] *** _despair86 <_despair86!~despair@2600:1700:8f61:163f:36e8:94ff:fede:7ce2> has quit IRC (Client Quit)
[03:57:21] *** despair86 <despair86!~despair@2605:6000:1515:45b:b62e:99ff:fea1:3bad> has quit IRC (Ping timeout: 250 seconds)
[03:57:22] *** _despair86 <_despair86!~despair@2600:1700:8f61:163f:36e8:94ff:fede:7ce2> has joined #illumos
[04:12:12] *** cartwright <cartwright!~chatting@gateway/tor-sasl/cantstanya> has quit IRC (Remote host closed the connection)
[04:14:11] *** cartwright <cartwright!~chatting@gateway/tor-sasl/cantstanya> has joined #illumos
[04:32:25] *** jcea <jcea!~Thunderbi@2001:41d0:1:8a82:7670:6e00:7670:6e00> has quit IRC (Remote host closed the connection)
[04:32:36] *** jcea <jcea!~Thunderbi@2001:41d0:1:8a82:7670:6e00:7670:6e00> has joined #illumos
[04:56:19] *** jcea <jcea!~Thunderbi@2001:41d0:1:8a82:7670:6e00:7670:6e00> has quit IRC (Quit: jcea)
[05:21:02] *** _despair86 <_despair86!~despair@2600:1700:8f61:163f:36e8:94ff:fede:7ce2> has quit IRC (Quit: brb installing patches)
[05:21:25] *** despair86 <despair86!~despair@2600:1700:8f61:163f:36e8:94ff:fede:7ce2> has joined #illumos
[06:03:20] *** nde <nde!~nde@c-98-253-7-24.hsd1.in.comcast.net> has joined #illumos
[06:07:37] *** nde <nde!~nde@c-98-253-7-24.hsd1.in.comcast.net> has quit IRC (Remote host closed the connection)
[07:24:26] *** insomnia <insomnia!~insomnia@shadowcat/actuallyamemberof/lollipopguild.insomnia> has quit IRC (Ping timeout: 240 seconds)
[07:36:27] *** insomnia <insomnia!~insomnia@shadowcat/actuallyamemberof/lollipopguild.insomnia> has joined #illumos
[07:36:38] *** lystra <lystra!~lystra@d53-64-11-169.nap.wideopenwest.com> has quit IRC (Quit: Leaving)
[07:49:17] *** lystra <lystra!~lystra@d53-64-11-169.nap.wideopenwest.com> has joined #illumos
[07:51:48] <tsoome> it will keep it sync
[08:43:12] *** alanc <alanc!~alanc@inet-hqmc02-o.oracle.com> has quit IRC (Remote host closed the connection)
[08:43:40] *** alanc <alanc!~alanc@inet-hqmc02-o.oracle.com> has joined #illumos
[09:21:37] *** arnoldoree <arnoldoree!~arnoldore@ranoldoree.plus.com> has joined #illumos
[10:02:51] *** andy_js <andy_js!~andy@94.11.180.149> has joined #illumos
[10:36:13] *** arnold_oree <arnold_oree!~arnoldore@ranoldoree.plus.com> has joined #illumos
[10:37:02] *** arnold_oree <arnold_oree!~arnoldore@ranoldoree.plus.com> has quit IRC (Remote host closed the connection)
[10:37:18] *** arnold_oree <arnold_oree!~arnoldore@ranoldoree.plus.com> has joined #illumos
[11:18:34] *** kerberizer <kerberizer!~luchesar@wikipedia/Iliev> has joined #illumos
[11:20:00] *** ed209 <ed209!~ed209@165.225.128.67> has quit IRC (Remote host closed the connection)
[11:20:06] *** ed209 <ed209!~ed209@165.225.128.67> has joined #illumos
[11:45:02] *** MilkmanDan <MilkmanDan!~dan@wilug/expat/MilkmanDan> has quit IRC (Ping timeout: 245 seconds)
[11:45:33] *** MilkmanDan <MilkmanDan!~dan@wilug/expat/MilkmanDan> has joined #illumos
[12:21:48] *** jperkin <jperkin!~jperkin@trumpton.perkin.org.uk> has quit IRC (Remote host closed the connection)
[12:24:04] *** jperkin <jperkin!~jperkin@trumpton.perkin.org.uk> has joined #illumos
[12:26:33] <gitomat> [illumos-gate] 11814 test-runner exit code should reflect test results -- John Levon <john.levon at joyent dot com>
[13:03:18] *** Manis <Manis!~Manis@84-73-20-163.dclient.hispeed.ch> has joined #illumos
[16:10:43] *** Qatz <Qatz!~db@2601:187:8400:5::42d> has quit IRC (Quit: Gone looking for beer)
[16:23:38] *** patdk-lap <patdk-lap!~patrickdk@208.95.164.6> has quit IRC (Ping timeout: 240 seconds)
[16:24:11] *** patdk-lap <patdk-lap!~patrickdk@208.95.164.6> has joined #illumos
[16:27:10] <tsoome> um, what can possibly keep dtrace module from unloading?
[16:27:55] <LeftWing> Have you done any tracing?
[16:29:15] <tsoome> attempted to run /opt/SUNWdtrt/bin/dtest -sa
[16:29:43] <tsoome> got this:
[16:29:43] <tsoome> root@beastie:/opt/SUNWdtrt# ./bin/dtest -a
[16:29:44] <tsoome> executing tests using /usr/sbin/dtrace ...
[16:29:45] <tsoome> testing /opt/SUNWdtrt/tst/common/aggs/tst.aggencoding.d ... 5 fffffffff80dd000 1ece0 60 1 dtrace (Dynamic Tracing)
[16:29:46] <tsoome> ERROR: couldn't unload dtrace
[16:31:48] <LeftWing> Huh
[16:32:08] <tsoome> and indeed, manual modunload fails as well
[16:32:19] <LeftWing> I'm not sure
[16:44:51] *** wl_ <wl_!~wl_@2605:6000:1b0c:5a5f::87c> has joined #illumos
[17:21:12] *** Qatz <Qatz!~db@2601:187:8400:5::c92> has joined #illumos
[17:34:22] <igork> LeftWing: about vioif and MTU=9k - what i need to check on hypervisor ? i have kvm host based on linux debian and try to check/test dilos as vm, but i can't use mtu=9k on vm. on host i have mtu=9k on ixgbe
[17:35:21] <igork> it was working with older virtio , but it is not working with latest in illumos
[17:35:29] <LeftWing> Did the Virtio Network device offer the MTU feature bit?
[17:35:39] <igork> how to check it ?
[17:35:54] <LeftWing> You'll want to look in the vioif_t
[17:36:13] <LeftWing> And at the feature bit stuff in vioif.h I believe
[17:36:25] <igork> ok, thanks, will try
[17:36:31] <LeftWing> Cool
[17:40:36] *** wiedi <wiedi!~wiedi@ip5b4096a6.dynamic.kabel-deutschland.de> has quit IRC (Read error: Connection reset by peer)
[17:41:11] *** wiedi <wiedi!~wiedi@ip5b4096a6.dynamic.kabel-deutschland.de> has joined #illumos
[18:04:56] *** arnold_oree <arnold_oree!~arnoldore@ranoldoree.plus.com> has quit IRC (Ping timeout: 240 seconds)
[18:33:44] <rmustacc> tsoome: Were you trying to run all the DTrace tests? -a doesn't run all tests, it tries to run the kernel anonymous DTrace bits, fwiw.
[18:34:15] <tsoome> oh my...
[18:34:22] <tsoome> :D
[18:34:42] <rmustacc> Just run dtest with no arguments.
[18:35:18] <rmustacc> So that's why it tried to unload the kernel module.
[18:36:25] <tsoome> yea… running already:D
[18:37:15] <rmustacc> There will probably be some failures that kind of end up happening regularly. If when it's done you run the dtfailures script and gist/pastebin/whatever it, send it to me and I can take a look.
[18:50:24] *** arnold_oree <arnold_oree!~arnoldore@ranoldoree.plus.com> has joined #illumos
[18:56:32] <KungFuJe1us> tsoome: looks like the new bootloader fixes still didn't fix the problem
[18:57:03] *** tsoome <tsoome!~tsoome@148-52-235-80.sta.estpak.ee> has quit IRC (Read error: Connection reset by peer)
[18:57:34] *** tsoome <tsoome!~tsoome@148-52-235-80.sta.estpak.ee> has joined #illumos
[18:58:23] <KungFuJe1us> tsoome: anything else I should try to make this bootloader work?
[18:58:39] <tsoome> ok, then we need to start digging...
[19:00:50] <KungFuJe1us> Let the digging begin :)
[19:08:16] *** tsoome <tsoome!~tsoome@148-52-235-80.sta.estpak.ee> has quit IRC (Read error: Connection reset by peer)
[19:09:57] *** tsoome <tsoome!~tsoome@148-52-235-80.sta.estpak.ee> has joined #illumos
[19:13:31] *** arnold_oree <arnold_oree!~arnoldore@ranoldoree.plus.com> has quit IRC (Remote host closed the connection)
[19:16:51] *** arnold_oree <arnold_oree!~arnoldore@ranoldoree.plus.com> has joined #illumos
[19:25:02] *** MilkmanDan <MilkmanDan!~dan@wilug/expat/MilkmanDan> has quit IRC (Ping timeout: 245 seconds)
[19:27:34] *** MilkmanDan <MilkmanDan!~dan@wilug/expat/MilkmanDan> has joined #illumos
[19:49:31] <tsoome> rmustacc: I did mail to advocates list in response about 11580 (Dan was asking)
[20:00:06] *** jcea <jcea!~Thunderbi@2001:41d0:1:8a82:7670:6e00:7670:6e00> has joined #illumos
[20:00:28] *** idodeclare <idodeclare!~textual@cpe-76-185-177-63.satx.res.rr.com> has joined #illumos
[20:20:49] <igork> LeftWing: https://share.getcloudapp.com/BluE6j9q
[20:20:59] <igork> it i part of vioif_t
[20:21:05] <igork> *it is
[20:21:36] <igork> what is feature flag what you pointed to check ?
[20:22:08] <LeftWing> The MTU one
[20:22:46] <igork> 0x5dc = 1500
[20:23:27] <LeftWing> That's the value, did the feature bit get negotiated
[20:23:59] <igork> https://paste.dilos.org/?7813f7d1ce452143#qFW3BB9JxDJp7DGgmInXKCM3Yj/qUT0w9gYLzM5F+uM=
[20:24:07] <igork> it is host
[20:24:29] <igork> where to check feature bit in vioif_t ?
[20:24:43] <igork> i'm on qemu console in kmdb
[20:26:03] <igork> also i can check something on host - it is dilos, but i checked smartos too - no mtu=9k available
[20:26:29] <LeftWing> You'll need to read vioif_attach and virtio_init
[20:26:46] <LeftWing> There are comments that describe the feature negotiation
[20:26:58] <igork> ok, let me try
[20:27:01] <LeftWing> I believe we record both what we negotiated and what the device initially offered
[20:27:50] <igork> vioif_attach() has no comment before function - maybe it is in another place?
[20:28:03] <igork> looking at file vioif.c
[20:29:20] <LeftWing> virtio.h also has a large block comment
[20:34:18] <igork> i have found a block about:
[20:34:18] <igork> if (vioif_has_feature(vif, VIRTIO_NET_F_MTU)) {}
[20:34:22] <igork> will try to check it
[20:34:37] <igork> it is in vioif_attach()
[20:39:02] <igork> LeftWing: https://paste.dilos.org/?a942b7f9ebe5f307#IFOGSaqmTl+n/zuKHqJEAZZRvtwxVfgv0rf+9M70MgM=
[20:39:07] <igork> vio_features = 0
[20:39:20] <igork> but we try to check it with 20
[20:39:30] <igork> fffffe00085718a0 vioif`vioif_has_feature+0xc(fffffe06490c5600, 20)
[20:39:44] <igork> return ((vio->vio_features & feature_mask) != 0);
[20:40:15] <igork> interest - virtio frame work broken in this case?
[20:46:36] <rmustacc> So the host needs to indicate that it supports changing the MTU according to the spec. If the feature isn't present, we don't know we can use a larger MTU.
[20:47:21] <igork> how to check it on host side with new virtio?
[20:47:35] <rmustacc> The way to check it on the host is no different than in the past.
[20:47:53] <igork> strange situation - it was working before update.
[20:48:00] <igork> ok, will try to check it on host
[20:48:36] <igork> host is supermicro X11 with xeon cpu
[20:48:55] <rmustacc> It's the hypervisor.
[20:49:03] <igork> yes
[20:52:51] *** freakazoid0223 <freakazoid0223!~matt@pool-96-227-98-169.phlapa.fios.verizon.net> has joined #illumos
[20:56:33] *** lblume <lblume!~lblume@greenviolet/laoyijiehe/lblume> has quit IRC (Quit: Leaving.)
[21:02:30] <LeftWing> I don't think you're printing the virtio_t properly
[21:02:38] <LeftWing> It shouldn't be Zero
[21:03:35] <rmustacc> igork: You printed a vioif_t as a virtio_t.
[21:03:39] <rmustacc> It isn't going to be correct.
[21:04:39] <rmustacc> With that argument you would need to use 'addr::print vioif_t vif_virtio | ::print virtio_t'
[21:05:02] <igork> in vioif_attache ?
[21:05:44] <rmustacc> Whenever makes sense. The thing you printed then was the wrong thing based on the addresses and everything you pasted here.
[21:05:52] <igork> ok
[21:06:00] <igork> i'll try to stop on virtio_init()
[21:06:20] <igork> virtio_feature_present()
[21:06:28] <igork> virtio_feature_present(virtio_t *vio, uint64_t feature_mask)
[21:06:33] <igork> will it correct place ?
[21:07:05] <igork> we try to check feature in vioif_attach
[21:07:37] <rmustacc> The features present and negotiationed should be constant for the lifetime of the device.
[21:07:43] <igork> 1624 if (vioif_has_feature(vif, VIRTIO_NET_F_MTU)) {)
[21:08:25] <igork> where will be better place to check it on boot time?
[21:08:45] <KungFuJe1us> hmm, should I be seeing xhci, failed to stop the controller?
[21:20:00] *** ed209 <ed209!~ed209@165.225.128.67> has quit IRC (Remote host closed the connection)
[21:20:07] *** ed209 <ed209!~ed209@165.225.128.67> has joined #illumos
[21:26:41] *** lblume <lblume!~lblume@2a01:e0a:31f:e0b0:7c14:8ac6:ed6c:526f> has joined #illumos
[21:26:44] *** idodeclare <idodeclare!~textual@cpe-76-185-177-63.satx.res.rr.com> has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…)
[21:27:02] *** lblume <lblume!~lblume@greenviolet/laoyijiehe/lblume> has quit IRC (Client Quit)
[21:27:26] *** lblume <lblume!~lblume@greenviolet/laoyijiehe/lblume> has joined #illumos
[21:45:01] *** oxford <oxford!~chatting@gateway/tor-sasl/cantstanya> has joined #illumos
[21:47:48] *** cartwright <cartwright!~chatting@gateway/tor-sasl/cantstanya> has quit IRC (Ping timeout: 260 seconds)
[21:51:37] *** gh34 <gh34!~textual@cpe-184-58-181-106.wi.res.rr.com> has joined #illumos
[21:52:12] *** gh34 <gh34!~textual@cpe-184-58-181-106.wi.res.rr.com> has quit IRC (Client Quit)
[22:27:51] *** arnold_oree <arnold_oree!~arnoldore@ranoldoree.plus.com> has quit IRC (Quit: Leaving)
[22:41:29] <igork> LeftWing: i know, it is hack and have to identify issue https://bitbucket.org/dilos/dilos-illumos/commits/c461bb0d273ad8b3c8a4391dc779ad339559f346
[22:41:47] <igork> but this test let me setup mtu=9k
[22:42:03] <igork> will try identify why i missed feature VIRTIO_NET_F_MTU
[22:42:14] <igork> just wanted let you know my experiment
[22:42:54] <igork> with this hack i can see on guest with viois:
[22:43:02] <igork> *vioif:
[22:43:02] <igork> vioif0       mtu             rw   1500           1500           60-65535
[23:20:30] *** arnoldoree <arnoldoree!~arnoldore@ranoldoree.plus.com> has quit IRC (Quit: Leaving)
[23:22:35] *** Manis <Manis!~Manis@84-73-20-163.dclient.hispeed.ch> has quit IRC (Ping timeout: 276 seconds)
[23:30:52] *** MilkmanDan <MilkmanDan!~dan@wilug/expat/MilkmanDan> has quit IRC (Ping timeout: 245 seconds)
[23:32:29] *** MilkmanDan <MilkmanDan!~dan@wilug/expat/MilkmanDan> has joined #illumos
top

   November 16, 2019  
< | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | >