Alfresco discussion and collaboration. Stick around a few hours after asking a question.
Official support for Enterprise subscribers: support.alfresco.com.
Join in the conversation by getting an IRC client and connecting to #alfresco at Freenode. Our you can use the IRC web chat.
More information about the channel is in the wiki.
More help is available in this list of resources.
2017-02-12 16:33:00 GMT <yreg> hey AFaust !
2017-02-12 16:33:10 GMT <AFaust> hey Younes
2017-02-12 16:33:42 GMT <yreg> I have been having a hard time configuring alfresco under windows !
2017-02-12 16:34:01 GMT <yreg> by configuring I meant tuning jvm params
2017-02-12 16:34:27 GMT <yreg> I have this setup with separate servers for Alfresco and Solr
2017-02-12 16:34:36 GMT <yreg> both windows
2017-02-12 16:34:57 GMT <yreg> Alfresco VM has 18G of ram and 4 cpu
2017-02-12 16:35:18 GMT <yreg> solr one has 48 G of ram and 8 cpu
2017-02-12 16:35:27 GMT <AFaust> Ok - so not locally but a "real" server
2017-02-12 16:36:00 GMT <yreg> Virtual machines on client's insfrastructure
2017-02-12 16:37:24 GMT <AFaust> And what specifically are the problems?
2017-02-12 16:37:44 GMT <yreg> I gave alfresco 14 G of ram and left 4 for the OS (we do connect from time to time to these servers using remote desktop to do all sorts of things)
2017-02-12 16:38:52 GMT <yreg> and gave solr 31 G (as I figured that any value >= 32G of RAM and <= 48G would have less performance than 31G
2017-02-12 16:39:27 GMT <yreg> well I am using the recommended GC config from alfresco and that seems to be not very optimal
2017-02-12 16:39:54 GMT <yreg> solr tomcat crashes from time to time, due to eden space being full
2017-02-12 16:40:43 GMT <yreg> that's one issue
2017-02-12 16:41:13 GMT <yreg> the other one is that I am now doing a full reindex
2017-02-12 16:41:56 GMT <AFaust> hehe - the recommended GC config from Alfresco is crap in my opinion
2017-02-12 16:41:57 GMT <yreg> and when I restart both servers the reindex process seems fast at first then starts to slow down
2017-02-12 16:42:06 GMT <AFaust> At least when it comes to Repository / Share tiers.
2017-02-12 16:42:36 GMT <AFaust> SOLR is another matter - the default isn't great for that but an alternative is not as clear cut
2017-02-12 16:42:38 GMT <yreg> tried to restart multiple times and I can confirm that whenever I restart the indexing is very fast at first and it gets slow after a while
2017-02-12 16:43:28 GMT <AFaust> Why do you figure that any value >= 326 and <= 48G would have less performance than 31 G?
2017-02-12 16:43:45 GMT <yreg> and I am thinking this should be related to the Garbage collection policy and jvm configuration
2017-02-12 16:44:20 GMT <yreg> AFaust, https://blog.codecentric.de/en/2014/02/35gb-heap-less-32gb-java-jvm-memory-oddities/
2017-02-12 16:44:22 GMT <alfbot> Title: Why 35GB Heap is Less Than 32GB - Java JVM Memory Oddities - codecentric AG Blog : codecentric AG Blog (at blog.codecentric.de)
2017-02-12 16:44:38 GMT <AFaust> The problem with large heap sizes is that CMS GC is very rigid/unflexible and a) effectively wastes a lot of memory in unused regions, and b) introduces larger "stop-world"-pauses
2017-02-12 16:45:56 GMT <AFaust> yreg: That is only because when you pass 32G heap size the compressed oops feature is disabled and memory addresses are the real 64 bit addresses they need to be
2017-02-12 16:46:32 GMT <yreg> exactly
2017-02-12 16:47:25 GMT <AFaust> So yes, there is overhead that will be added when going to and above 32 G, but the effect is dependent on the structure / size of objects that are instantiated...
2017-02-12 16:48:07 GMT <AFaust> The example code linked to in the blog article is very aggressive in terms of forcing expensive object pointers
2017-02-12 16:49:10 GMT <AFaust> Anyway
2017-02-12 16:49:20 GMT <AFaust> Have you considered using the G1 GC algorithm?=
2017-02-12 16:50:20 GMT <AFaust> At least for Repository/Share it is much more efficient.... there is no more strict sizing for Eden, Survivor and Old Generation (as long as you don't force it), so issues with these should not occur anymore
2017-02-12 16:50:26 GMT <yreg> I am willing to experiment with it at least to see if it is any better
2017-02-12 16:50:55 GMT <yreg> the issue with eden space only occured on the solr side
2017-02-12 16:51:08 GMT <AFaust> For SOLR, some people claim that G1 GC can have side-effects in certain situations. Unfortuantely I have not yet had a chance to verify this with a production-grade system.
2017-02-12 16:51:25 GMT <yreg> and after tomcat crashed, it kept krashing whenever I restarted the service
2017-02-12 16:51:42 GMT <yreg> I ended up restarting the VM for it to work correctly again
2017-02-12 16:52:22 GMT <AFaust> Regarding SOLR + CMS - I usually have JVM settings that are adapted to increase relative size of eden/survivor in comparison to old generation, as most of memory in SOLR is usually needed for per-query data structures
2017-02-12 16:52:52 GMT <AFaust> -XX:NewRatio=1 -XX:SurvivorRatio=2 -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=80
2017-02-12 16:53:07 GMT <AFaust> ^^ my standard JVM settings for SOLR in addition to Xmx/Xms
2017-02-12 16:54:24 GMT <AFaust> Whhat are the settings you are using? Which of the various Alfresco documentations / blog post entries did you use as reference?
2017-02-12 16:54:25 GMT <yreg> Thanks for sharing this, will give it a run on this system and let you know how that goes !
2017-02-12 16:54:53 GMT <yreg> wait a sec, it was a wolleague who configured the servers
2017-02-12 16:56:27 GMT <yreg> -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ReservedCodeCacheSize=128m -Xmx=31744m -Xms=31744m
2017-02-12 16:59:42 GMT <yreg> I do not think I should be increasing Xmx and Xms to more than 32 G right now
2017-02-12 17:05:42 GMT <yreg> I do not have accurate measures sustaining this
2017-02-12 17:06:31 GMT <AFaust> I never understood the benefit of setting -XX:+UseParNewGC and -XX:ReservedCodeCacheSize
2017-02-12 17:06:53 GMT <AFaust> And I never encountered any issues not setting these (I have never used these)
2017-02-12 17:06:56 GMT <yregaieg> but I got the impression (from the graphs in our monitoring stack) that 31G worked better during reindex
2017-02-12 17:07:24 GMT <yregaieg> but this definitely deserve some deeper analysis
2017-02-12 17:07:33 GMT <AFaust> For SOLR I can understand how the OOP issue with 31G vs 32+G might be an issue
2017-02-12 17:07:58 GMT <AFaust> Because SOLR uses a lot of very small objects internally...
2017-02-12 17:08:29 GMT <AFaust> I still have to look up and understand how primitives / arrays of primitives work with OOP
2017-02-12 17:09:19 GMT *** yregaieg is now known as yreg
2017-02-12 17:12:33 GMT <AFaust> hehe UseParNewGC deprecated in JDK 8 and removed in JDK 9
2017-02-12 17:13:16 GMT <AFaust> damn - the guy who wrote that did not express himself very clearly
2017-02-12 17:13:23 GMT <yreg> Makes sense, by analysing graphs for heap size over time
2017-02-12 17:13:33 GMT <yreg> I get the impression that it is not effective
2017-02-12 17:13:46 GMT <AFaust> UseParNewGC is not deprecated in general - the guy attributed the remarks to the implicit use of CMS which is deprecated in 8 and removed in 9
2017-02-12 17:15:19 GMT <yreg> For alfresco side, what would be a typical config ?
2017-02-12 17:15:21 GMT <yreg> -XX:+UseG1GC -XX:MaxGCPauseMillis=250
2017-02-12 17:15:22 GMT <yreg> ?
2017-02-12 17:16:32 GMT <AFaust> Something like that, yes. I typically don't even bother with the max pause setting
2017-02-12 17:16:43 GMT <AFaust> The default value is quite reasonable
2017-02-12 17:17:40 GMT <AFaust> And since caches can be configured to proper sizes based on load tests / metrics to avoid exchausting the heap there should rarely be a case when longer GC pauses happen
2017-02-12 17:18:15 GMT <AFaust> Too bad it is the Lucene folk who warn against G1GC, even though SOLR folk don't seem to have ever encountered an issue with it
2017-02-12 17:18:15 GMT <AFaust> https://wiki.apache.org/solr/ShawnHeisey#G1_.28Garbage_First.29_Collector
2017-02-12 17:18:16 GMT <alfbot> Title: ShawnHeisey - Solr Wiki (at wiki.apache.org)
2017-02-12 17:19:12 GMT <AFaust> Lately I have started to set G1HeapRegionSize
2017-02-12 17:19:37 GMT <AFaust> Especially if I use a Xms that is not equal to Xmx
2017-02-12 17:20:08 GMT <AFaust> Because the default value is calculated based on Xms, which might not be ideal when the heap has grown to Xmx
2017-02-12 17:21:08 GMT <yreg> interesting
2017-02-12 17:22:20 GMT <yreg> Thank you for these valuable thoughts !
2017-02-12 17:22:21 GMT <AFaust> Some of the implications of G1GC and G1HeapRegionSize are also interesting for the question "when to shard"...
2017-02-12 17:23:26 GMT <yreg> hmm, that is not applicable in this case, but definitely interesting to know
2017-02-12 19:17:13 GMT <rootinshell> Hello eveyone
2017-02-12 19:17:21 GMT <rootinshell> got a question :)
The other logs are at http://esplins.org/hash_alfresco