[00:00:12] *** alesj has joined #jbosstesting
[00:07:30] <jc3> where does output.log come from?  does arquillian override log4j config somehow?
[00:12:34] *** bobmcw has quit IRC
[00:41:03] *** rruss has quit IRC
[00:42:50] *** alesj has quit IRC
[00:44:32] *** timte has quit IRC
[00:55:30] *** michaelschuetz has joined #jbosstesting
[01:05:19] *** bleathem has quit IRC
[01:21:38] *** ALR has joined #jbosstesting
[01:53:37] *** ldimaggi has joined #jbosstesting
[03:31:33] *** michaelschuetz has quit IRC
[04:01:34] *** bleathem has joined #jbosstesting
[04:27:59] <jbossbot> git [shrinkwrap] push master 0660e20.. Andrew Lee Rubinger [SHRINKWRAP-246] Clear usage of TCCL; draw association in the Configuration to use an explicit CL instead for archive extension type loading
[04:27:59] <jbossbot> jira [SHRINKWRAP-246] Shrinkwrap does not work in a modular environment [Open (Unresolved) Bug, Critical, Unassigned] https://issues.jboss.org/browse/SHRINKWRAP-246
[04:28:00] <jbossbot> git [shrinkwrap] push master URL: http://github.com/shrinkwrap/shrinkwrap/compare/7ac8c91...0660e20
[04:37:04] *** rruss has joined #jbosstesting
[04:57:59] *** lightguard_jp has quit IRC
[06:07:54] *** ldimaggi has quit IRC
[07:13:53] *** oskutka has joined #jbosstesting
[07:19:32] *** lightguard_jp has joined #jbosstesting
[07:37:41] *** timte has joined #jbosstesting
[07:43:44] *** bleathem has quit IRC
[08:07:11] *** jharting has joined #jbosstesting
[08:14:07] *** PeteRoyle has quit IRC
[08:32:47] *** ge0ffrey has joined #jbosstesting
[08:50:26] *** lightguard_jp has quit IRC
[08:54:43] *** maschmid has joined #jbosstesting
[08:54:45] *** mgoldmann has joined #jbosstesting
[08:55:23] *** wolfc has joined #jbosstesting
[09:04:23] *** rruss has quit IRC
[09:27:36] *** jeand has joined #jbosstesting
[09:38:40] *** kpiwko has joined #jbosstesting
[09:44:46] *** wyer has joined #jbosstesting
[09:49:17] *** jeand has quit IRC
[09:49:17] *** jeand has joined #jbosstesting
[09:56:39] *** maeste has joined #jbosstesting
[10:12:44] *** Jaikiran has joined #jbosstesting
[10:30:42] *** Jaikiran is now known as Jaikiran|Lunch
[10:49:56] *** alesj has joined #jbosstesting
[11:14:46] *** Elisha has quit IRC
[11:17:01] *** vtunka has joined #jbosstesting
[11:17:22] *** Elisha has joined #jbosstesting
[11:44:13] *** Jaikiran|Lunch has quit IRC
[11:45:12] *** Jaikiran has joined #jbosstesting
[12:15:18] <ALR> wolfc: Lucy, you got some 'splaining to do.  https://github.com/wolfc/shrinkwrap/commit/3d4969db684d1ce5183cb4ca11a8b4ea9c9da213 :P
[12:15:19] <jbossbot> git [shrinkwrap] 3d4969d.. Carlo de Wolf SHRINKWRAP-269: test case
[12:15:20] <jbossbot> jira [SHRINKWRAP-269] Shrinkwrap is hanging in JdkZipExporterDelegate [Reopened (Unresolved) Bug, Major, Andrew Rubinger] https://issues.jboss.org/browse/SHRINKWRAP-269
[12:15:43] <wolfc> hehe
[12:16:09] <wolfc> ALR, the design is bad. In VFS there is an example of a virtual JarInputStream.
[12:16:34] <ALR> Listening...
[12:16:46] <wolfc> I want that one in ShrinkWrap, instead of the Pipe/Executor/Thread thingy.
[12:17:17] <ALR> What is the problem with the pipe?
[12:17:33] <ALR> I mean, this test case violates some design principles itself.
[12:17:38] <ALR> First, Thread.sleep
[12:17:41] <wolfc> What the test case shows, if I loose the InputStream it leaves garbage.
[12:17:59] <ALR> Second, attempting a request to run gc (and only once at that)
[12:18:23] <ALR> Is that even valid?  I mean you set the instream to null...
[12:18:49] <ALR> So I suppose it'll lock up and never complete...
[12:18:53] <wolfc> The test case is based on lucky guesses. I'm good at those. :-P
[12:19:00] <ALR> As you need to read the instream fully for the operation to complete
[12:19:01] <wolfc> https://github.com/jbossas/jboss-vfs/blob/master/src/main/java/org/jboss/vfs/VirtualJarFileInputStream.java
[12:20:32] <ALR> wolfc: This is only valid for ZIP.
[12:22:35] <wolfc> and its companion https://github.com/jbossas/jboss-vfs/blob/master/src/main/java/org/jboss/vfs/VirtualJarInputStream.java
[12:22:45] <ALR> wolfc: I'm still analyzing this test case.
[12:22:57] <ALR> We never read "in", so resetting it to null has no effect.
[12:23:11] <ALR> Same with numThreads and numThreads2...never read
[12:23:12] <wyer> it made my brain implode
[12:23:38] <wyer> but to be fair my brain has imploded a lot today :P
[12:23:58] <wolfc> then it has probably turned into a black hole by now :-D
[12:24:09] <ALR> Though the numThreads don't matter so much, you're more interested in the contents of the tarray vars
[12:24:23] <ALR> wolfc: So what are you trying to do by setting in to null?
[12:24:52] <wolfc> It trigger the gc (to some extend, lets not go into those details)
[12:24:53] <ALR> Just make it eligible for GC?
[12:25:00] <ALR> It *may*, maybe...
[12:25:10] <wolfc> yeah, yeah, details :-)
[12:25:54] <ALR> And finally:
[12:26:08] <ALR> You're expecting the output of Thread.enumerate to be idempotent
[12:26:35] <ALR> Meaning that if you execute Thread.enumerate twice in succession...you're expecting the results to be equal.
[12:26:45] <ALR> And that's simply not going to be true.
[12:28:01] <wolfc> Could be, I was also contemplating another test with an user supplied executor
[12:28:45] <wolfc> Again loose the InputStream (because that is what's happening in AS) and then try to export another archive. You would effectively hang.
[12:29:05] <ALR> Yep, that I'll do.
[12:29:39] <wolfc> But such a test is moot, because it should never use an executor.
[12:29:48] <ALR> You're saying that the design here will hang if the InputStream is not fully read, correct?
[12:30:17] <ALR> ie. If you attempt a second export without completing the first?
[12:30:26] <ALR> (In a SingleThreadExecutor)
[12:30:28] <wolfc> I'm saying that it leaves artifacts.
[12:30:52] <wyer> wolfc: its just a neutron star atm, blackhole soon tho :P
[12:31:51] <wolfc> Lunch!
[12:31:52] <ALR> wolfc: What leaves artifacts?
[12:32:00] <ALR> OKee.
[12:51:07] <ALR> wolfc: For when you're back:
[12:51:36] <ALR> I'd like to see a test which shows the real problem.  This one here is asking what Threads are active.
[12:51:39] <wolfc> I shoved my lunch in
[12:51:44] <ALR> http://pastebin.com/zGNj5kh8 < For instance, though this one passes
[12:51:47] <ALR> Ah
[12:52:37] <wolfc> The real problem is that garbage is left behind. I'll write another test, hold on.
[12:52:50] <ALR> wolfc: What garbage?
[12:52:59] <wolfc> the export thread
[12:53:18] <ALR> wolfc: I have to ask: so?
[12:54:09] <wolfc> ALR: so if we have a long running process the garbage mounts up until we go OOM
[12:54:27] <ALR> No way.
[12:54:37] <ALR> The export thread will fill the buffer
[12:54:40] <ALR> Then block
[12:55:00] <ALR> No OOM, the memory consumption will be limited to the buffer size
[13:04:38] <wolfc> each export creates an executor
[13:04:53] <ALR> That is true.
[13:05:12] <ALR> *Unless if you are in a Domain where you made your own Executor.
[13:05:16] <ALR> Or specified it
[13:05:21] <ALR> AS should be one of these cases
[13:05:27] <ALR> As AS is a runtime environment
[13:05:35] <wolfc> In a Domain it'll hang: http://fpaste.org/aQTW/
[13:06:31] <ALR> wolfc: Now that's what I'm getting at.
[13:07:12] <wolfc> ALR, in AS it'll even be worse if we would use a single thread pool
[13:08:19] <ALR> wolfc: YOu don't even need the GC or anything here.
[13:08:28] <ALR> wolfc: Very simply:
[13:08:38] <wolfc> the assumption that people will close your input stream is wrong. It'll only work out in a perfect world.
[13:08:47] <ALR> You cannot have more active export operations than Threads in the Executor
[13:09:12] <wolfc> The use of an Executor is wrong.
[13:09:54] <ALR> wolfc: http://pastebin.com/UCDURniQ
[13:10:19] <ALR> wolfc: Change line 4 to have a fixed size of 2, and all works.
[13:10:24] * ALR thinking on this.
[13:10:56] <ALR> The executor is in place for 2 reasons:
[13:11:27] <ALR> 1) Make the export process a "pull"; don't flood the processor if nothing is reading the result
[13:11:57] <ALR> 2) Limit the RAM impact to a buffer, don't just export the whole thing and hold the result in memory
[13:12:45] <ALR> wolfc: How is AS even running into this?  When a Domain with no ExecutorService is supplied, it's a new ExecutorService for every export call.
[13:14:27] *** ldimaggi has joined #jbosstesting
[13:14:46] <wolfc> ALR, in extreme conditions AS won't close the InputStream
[13:15:09] <ALR> wolfc: Yes, but even still....
[13:15:27] <ALR> wolfc: Subsequent export requests would use a new ExecutorService entirely.
[13:15:42] <wolfc> ALR, that constitutes a memory leak
[13:15:48] <ALR> So it's not like they'd be waiting for an available executor
[13:16:20] <wolfc> Ultimo everybody needs to ask for a jboss-threads pool
[13:16:20] <ALR> wolfc: Again, the memory leak is limited to the size of the buffer.
[13:16:31] <wolfc> +executor service + thread
[13:16:50] <ALR> wolfc: Yes, AS should provide a JBoss Threads ExecutorService to the ShrinkWrap Domain
[13:16:54] <ALR> But that notwithstanding:
[13:17:07] <ALR> You ask for an InputStream, it's your responsibility to close it.
[13:17:13] <ALR> wolfc: For instance:
[13:17:16] <ALR> URL.openStream
[13:17:24] <wolfc> In a perfect world, yes
[13:17:31] <ALR> You think you can just keep on opening those without closing them?
[13:17:54] <ALR> By the same nature, that's a memory leak.  You'd call that a bug in java.net.URL?
[13:18:16] <ALR> It's the responsibility of the caller to close an opened instream.
[13:18:31] <ALR> If you lose it, I expect you to lose the resources.
[13:19:47] *** jharting has quit IRC
[13:22:19] <wolfc> Actually, you won't
[13:24:09] <wolfc> If the GC does as advertised the resources will be gone (lots of assumptions and details barring).
[13:25:44] <ALR> wolfc: The problem you reported was a hang
[13:25:46] <ALR> Not a memory leak.
[13:25:55] <ALR> And we haven't reproduced that hang.
[13:26:08] <wolfc> I just did
[13:26:18] <wolfc> http://fpaste.org/aQTW/
[13:26:18] <ALR> wolfc: Nope, you reproduced a different hang.
[13:26:31] <ALR> That one uses an explicit ExecutorService
[13:26:54] <ALR> Where the number of executors is less than the number of export tasks you're asking it to do.
[13:26:57] <wolfc> It shows that the implementation is flawed.
[13:27:10] <ALR> Ha, that's just bad config!
[13:27:20] <ALR> You're sending 2 jobs to only one worker!
[13:27:25] <ALR> What'd you expect it to do?
[13:27:30] <wolfc> No, because job 1 is finished.
[13:27:38] <wolfc> (lost in space :-) )
[13:27:52] <ALR> Job 1 isn't finished until it's done
[13:27:58] <ALR> And it's not done unless completely read.
[13:28:01] <ALR> That is the design.
[13:28:05] <ALR> "pull" model.
[13:28:33] <wolfc> The design is not bastard proof.
[13:28:50] *** Elisha has quit IRC
[13:29:05] <ALR> wolfc: OK, then let me invert the problem:
[13:29:18] <ALR> How would you fix it, without flooding the CPU or RAM?
[13:29:28] <ALR> That's what the "pull" design buys us.
[13:29:39] <ALR> As tested by the ZipExporterStressTestCase
[13:29:58] <ALR> You can export archives of greater size than available RAM.
[13:30:18] <ALR> The whole process used to be synchronous export, BTW
[13:30:34] <ALR> But this kept the whole output in memory.
[13:30:47] *** Elisha has joined #jbosstesting
[13:30:59] <ALR> The pull model we have now means that we only export as the reader is reading, and the RAM impact is limited to a buffer.
[13:31:19] <ALR> So if it's OOM we're worried about, the current thing is much safer.
[13:33:20] <ALR> wolfc: Hehe, if you're worried about the blocking, configure JBoss EJB3 SLSBs for a fixed pool size of 10, then issue 11 requests to it.
[13:33:34] <ALR> You'd think a user was crazy for blaming our EJB3 impl for blocking. :)
[13:34:18] <wolfc> if the SLSB instance itself would block in some ejb3 code, then it would be our problem. Which is what's happening on SW
[13:35:40] <ALR> wolfc: I have a feeling all we need is docs.
[13:35:47] <ALR> On exportAsInputStream
[13:36:02] <ALR> instructing the user that the operation will not complete until the stream is fully read
[13:37:45] <ALR> wolfc: Unless you can solve this design in a better way
[13:37:55] <ALR> Which also addresses the ZipExporterStressTestCase
[13:42:02] *** vtunka has quit IRC
[13:42:31] <wolfc> You'll owe me lots of Baco :-)
[13:43:09] <ALR> wolfc: Hehe, well let's talk it through before you invest too much time
[13:43:18] <ALR> I thought about this particular problem a lot
[13:43:46] <ALR> So I'm comfortable putting some constraints on the user...ie. fully-read the InStream
[13:43:46] *** vtunka has joined #jbosstesting
[13:44:05] <ALR> To make the thing work in a nicely buffered fashion that won't flood RAM or the CPU
[13:59:35] *** ALR has quit IRC
[14:24:38] *** bobmcw has joined #jbosstesting
[14:31:51] *** bobmcw has quit IRC
[14:32:25] *** rruss has joined #jbosstesting
[14:33:24] *** rruss has quit IRC
[14:35:56] *** bobmcw has joined #jbosstesting
[15:05:58] *** oskutka has quit IRC
[15:15:39] *** ALR has joined #jbosstesting
[15:39:14] *** dblevins has joined #jbosstesting
[15:48:15] *** jharting has joined #jbosstesting
[16:02:20] *** timte has quit IRC
[16:16:48] *** aslak has joined #jbosstesting
[16:16:48] *** aslak has quit IRC
[16:16:48] *** aslak has joined #jbosstesting
[16:26:04] *** aslak has quit IRC
[16:26:55] *** aslak has joined #jbosstesting
[16:43:29] *** aslak has quit IRC
[16:44:11] *** aslak has joined #jbosstesting
[16:48:11] *** wyer_ has joined #jbosstesting
[16:48:40] *** wyer has quit IRC
[16:50:07] <ALR> aslak: Ping
[16:51:05] *** aslak has quit IRC
[16:51:10] *** aslak has joined #jbosstesting
[16:51:18] <aslak> ALR, heya
[16:51:22] <aslak> kpiwko, ping
[16:51:36] <ALR> aslak: Yo.  Where are you hiding the new AS7 ARQ stuff?
[16:51:49] <kpiwko> ALR, aslak: hey!
[16:52:04] <ALR> kpiwko: Yo.
[16:52:04] <aslak> ALR, https://github.com/aslakknutsen/arquillian-container-jbossas/tree/JBossAS7
[16:52:20] <aslak> kpiwko, hows was the drive home?
[16:52:42] <kpiwko> aslak: quick but sleepy...what was yours?
[16:52:53] *** wyer_ is now known as wyer
[16:53:05] <ALR> aslak: I'll move this to the AS7 tree, k?
[16:53:11] <kpiwko> aslak: think I got a coffee resistance at confess
[16:53:13] <aslak> ALR, sure
[16:53:20] <ALR> aslak: And it all works / is tested?
[16:53:36] <aslak> ALR, it has been publicly demoed with success.. :)
[16:53:39] <aslak> it's
[16:53:47] <ALR> aslak: Superduper.
[16:53:49] <aslak> in two countries
[16:54:05] <ALR> aslak: I only care about the U.S.
[16:54:21] <aslak> ALR, fine someone to show it to then.. :)
[16:54:26] <aslak> find
[16:54:40] <ALR> aslak: :D
[16:55:16] <aslak> kpiwko, ok flight. nothing special :)
[16:56:25] <ALR> aslak: Oh, and those will work with Alpha5?
[16:56:34] <aslak> ALR, those are alpha5
[16:56:42] <ALR> aslak: ie. They don't depend upon anything in the main ARQ tree introduced after?
[16:56:43] <ALR> Super.
[16:56:56] <aslak> aa, right.. no
[16:56:56] <ALR> Oh whoops, it's its own repo.  great
[16:57:21] <ALR> aslak: I should pull from yours, or the upstream?
[16:57:23] <aslak> it's setup against core SNAPSHOT, but nothing new in there
[16:57:40] *** jharting has quit IRC
[16:58:37] <aslak> ALR, hmm, not sure if we want to do the same as we did with Weld here?
[16:58:51] <ALR> aslak: Wht'd we do with Weld?
[16:59:05] <aslak> where Arq has one v. that supports the latest Arq, and AS7 has a v that supports the latests AS7 using what ever Arq release they need
[17:00:08] <aslak> i think since the time frame for final is very short, and that means locked api/spis.. we can keep it in the as7 tree for now
[17:00:25] <aslak> we can pull it out and release a different arq v if needed at a later state
[17:00:29] <ALR> aslak: I think the AS7 tree makes the most sense looking forward.
[17:04:34] <aslak> ALR, just pull them from my branch. you probably want to change some stuff in there, e.g. groupid etc. if you take over i'll just nuke my branch
[17:05:31] <ALR> aslak: Yeah.
[17:05:39] <ALR> AAaaaaannd, building fire alarm.  Greeeeeat
[17:05:45] <ALR> Back when I can see what's up I gues
[17:05:49] <aslak> :)
[17:05:50] <ALR> Ooooh, it's off.
[17:05:51] <ALR> Nice
[17:05:52] <kpiwko> :)
[17:05:56] <ALR> Phew
[17:06:04] <ALR> Fire must have wiped out the alarm. :D
[17:12:18] *** dblevins has quit IRC
[17:17:15] *** dblevins has joined #jbosstesting
[17:18:46] <ALR> aslak: Great example of why these need to go to one place : https://github.com/jbossas/jboss-as/commit/158b82a8b331c3d22d4afed9ba34de78a26ff351
[17:18:47] <jbossbot> git [jboss-as] 158b82a.. bstansberry at jboss dot com [JBAS-9312] Don't proceed with tests until managed server is fully started
[17:18:48] <jbossbot> jira [JBAS-9312] JBossAsManagedContainer prematurely thinks the AS is started [Resolved (Done) Bug, Major, Brian Stansberry] https://issues.jboss.org/browse/JBAS-9312
[17:18:54] *** alesj has left #jbosstesting
[17:19:00] *** OndejZizka has joined #jbosstesting
[17:20:22] <aslak> :)
[17:25:10] *** ge0ffrey has quit IRC
[17:35:14] <ALR> aslak: For the AS server to start once at the beginning of the suite, forkMode should be...once ?
[17:40:26] *** maeste has quit IRC
[17:42:34] *** lightguard_jp has joined #jbosstesting
[17:56:51] *** wyer has quit IRC
[18:06:46] <ALR> aslak: And the remote AS7 test case is hanging here
[18:07:41] *** vtunka has quit IRC
[18:13:04] <aslak> ALR, once or never
[18:13:22] <ALR> aslak: And the remote case doens't hang for you?
[18:13:38] <aslak> ALR, the testcases in remote should be set to skipTests by default
[18:13:48] <aslak> you need to start the container for them to run
[18:17:49] *** maschmid has quit IRC
[18:20:14] <ALR> aslak: DUH.
[18:20:19] <ALR> My, I'm a moron.
[18:30:01] *** kpiwko has quit IRC
[18:30:42] <aslak> hehe
[18:31:12] <aslak> ALR, not sure why they hang forever tho. might be some missing timout set in the as client lib
[19:25:42] *** Jaikiran has quit IRC
[20:25:25] *** alesj has joined #jbosstesting
[20:29:51] *** bleathem has joined #jbosstesting
[21:03:32] *** rruss has joined #jbosstesting
[21:04:00] *** rruss has quit IRC
[21:15:26] *** lightguard_jp has quit IRC
[21:17:01] *** ALR has quit IRC
[21:25:56] *** mgoldmann has quit IRC
[21:28:04] *** lightguard_jp has joined #jbosstesting
[21:36:35] *** lightguard_jp has quit IRC
[21:45:22] *** ALR has joined #jbosstesting
[21:52:42] *** lightguard_jp has joined #jbosstesting
[21:59:04] <jbossbot> git [descriptors] push master 9e71662.. Lincoln Baxter, III Added JSF Suffixes and Mappings settings
[21:59:04] <jbossbot> git [descriptors] push master URL: http://github.com/shrinkwrap/descriptors/compare/e6eac70...9e71662
[21:59:37] *** jdlee has quit IRC
[22:18:33] *** ALR has quit IRC
[22:42:14] *** timte has joined #jbosstesting
[22:46:46] *** ldimaggi has quit IRC
[22:49:54] *** ALR has joined #jbosstesting
[22:49:58] *** jeand has quit IRC
[22:51:03] *** jdlee has joined #jbosstesting
[22:51:03] *** jdlee has joined #jbosstesting
[22:57:36] *** jdlee has quit IRC
[22:59:38] *** maeste has joined #jbosstesting
[23:00:04] *** oskutka has joined #jbosstesting
[23:01:22] *** michaelschuetz has joined #jbosstesting
[23:14:11] *** oskutka has quit IRC
[23:15:43] *** jdlee has joined #jbosstesting
[23:15:43] *** jdlee has joined #jbosstesting
[23:16:55] *** alesj has quit IRC
[23:18:58] *** ALR has quit IRC
[23:24:21] *** ALR has joined #jbosstesting
[23:51:28] *** timte has quit IRC





top