Daily Log for #alfresco IRC Channel

Alfresco discussion and collaboration. Stick around a few hours after asking a question.

Official support for Enterprise subscribers: support.alfresco.com.

Joining the Channel:

Join in the conversation by getting an IRC client and connecting to #alfresco at Freenode. Our you can use the IRC web chat.

More information about the channel is in the wiki.

Getting Help

More help is available in this list of resources.

Daily Log for #alfresco

2017-03-10 00:03:56 GMT <AFaust> ~later tell yreg: Distracted by the YouTubes... "Don't use MS SQL" is most important. Otherwise: make sure query plans are recompiled otherwise you may get horrible performance, especially for DB FTS

2017-03-10 00:03:56 GMT <alfbot> AFaust: The operation succeeded.

2017-03-10 08:22:52 GMT <alfbot> yreg: Sent 8 hours and 18 minutes ago: <AFaust> Distracted by the YouTubes... Don't use MS SQL is most important. Otherwise: make sure query plans are recompiled otherwise you may get horrible performance, especially for DB FTS

2017-03-10 09:07:12 GMT <yreg> Morning guys

2017-03-10 13:29:25 GMT <douglascrp> morning guys

2017-03-10 13:37:53 GMT <bhagyas> douglascrp: Good morning

2017-03-10 13:37:59 GMT <bhagyas> AFaust: Did you update the Notepad++?

2017-03-10 13:41:00 GMT <AFaust> bhagyas: Why are you asking?

2017-03-10 13:41:32 GMT <bhagyas> https://news.ycombinator.com/item?id=13824032

2017-03-10 13:41:33 GMT <alfbot> Title: Notepad++ V 7.3.3 – Fix CIA Hacking Notepad++ Issue | Hacker News (at news.ycombinator.com)

2017-03-10 13:41:40 GMT <bhagyas> :P

2017-03-10 13:44:05 GMT <AFaust> Well - I don't use it for much...

2017-03-10 13:57:39 GMT <mrks_js> what the

2017-03-10 14:03:46 GMT <yreg> bhagyas, what is this vuln doing?

2017-03-10 14:03:59 GMT <yreg> seems like a long discussion to read :P

2017-03-10 14:06:16 GMT <yreg> AFaust, the ibatis caches (or the way they are implemented in alfresco) are very dump

2017-03-10 14:06:23 GMT <yreg> dumb*

2017-03-10 14:07:59 GMT <yreg> they are using new objects for the same query strings 1000+ character each, as for the number as many nodes as you have loaded in the current transaction, in my case 500,000+

2017-03-10 14:09:17 GMT <yreg> and not only query strings are duplicated, lots of other stuff, and in those caches, the key is actually bigger than the value

2017-03-10 14:10:25 GMT <AFaust> yreg: Are the Strings really duplicated? Or are you just seeing multiple String that reference to the same char-array?

2017-03-10 14:10:56 GMT <AFaust> But that sounds really problematic... I'll have to do a local test and heap analysis to verify...

2017-03-10 14:11:19 GMT <AFaust> Problem with the ibatis queries: They contain a heck of a lot whitespace characters loaded from the XML files

2017-03-10 14:12:02 GMT <yreg> and those are not even trimmed

2017-03-10 14:12:12 GMT <yreg> before including the queries in the cache keys

2017-03-10 14:12:17 GMT <AFaust> There should be an option / feature that automatically reduces multiple whitespaces to a single one

2017-03-10 14:12:36 GMT <AFaust> Trimming is not the problem - it is more about compression

2017-03-10 14:17:55 GMT <yreg> AFaust, here it is : http://imgur.com/a/SJn7B

2017-03-10 14:17:56 GMT <alfbot> Title: Imgur: The most awesome images on the Internet (at imgur.com)

2017-03-10 14:18:07 GMT <yreg> the references seem different to me

2017-03-10 14:19:21 GMT <yreg> and that object is almost 968 chars long, duplicated 517,803 times

2017-03-10 15:10:55 GMT <yreg> Does any one know of the LDAP synchronization attempting to connect to LDAP server with wrong username/pass before trying the configured values ?

2017-03-10 15:13:21 GMT <AFaust> yreg: LDAP is performing a couple of checks at subsystem startup that use hard-coded auth-schemes - basically to check if some insecure options are enabled on LDAP and info/warn about that

2017-03-10 15:13:40 GMT <AFaust> But synchronisation should then be with configured values only

2017-03-10 15:16:00 GMT <yreg> hmm that makes sense

2017-03-10 15:16:40 GMT <yreg> we have an issue with alfresco being locked out of LDAP because of multiple failing auth attemptd

2017-03-10 15:18:37 GMT <yreg> probably due to multiple servers being restarted about the same time

2017-03-10 15:19:05 GMT <AFaust> yreg: All handled here - https://github.com/Alfresco/community-edition-old/blob/master/projects/repository/source/java/org/alfresco/repo/security/authentication/ldap/LDAPInitialDirContextFactoryImpl.java#L478

2017-03-10 15:19:06 GMT <alfbot> Title: community-edition-old/LDAPInitialDirContextFactoryImpl.java at master · Alfresco/community-edition-old · GitHub (at github.com)

2017-03-10 15:20:01 GMT <yreg> thanks

2017-03-10 15:20:24 GMT <AFaust> Would be super-annoying if these connections (which are not documented anywhere) would be causing the server (IP) to be locked out...

2017-03-10 15:21:12 GMT <AFaust> That reminds me - maybe I'll override this behaviour in my MT LDAP implementation to make it at least configurable...

2017-03-10 15:23:36 GMT <AFaust> yreg: Ok - just done my own tests and confirmed the duplicate String stuff...

2017-03-10 15:24:38 GMT <AFaust> It can be annoying especially for empty result sets, e.g. I am currently running a DB FTS query that does not select anything and cache key is ~ 3KiB while response value is 32 byte

2017-03-10 15:28:40 GMT <yreg> Told you so <grin>

2017-03-10 15:29:08 GMT <yreg> AFaust, I added config to disable ibatis caches

2017-03-10 15:29:19 GMT <yreg> will share that config if it actually works

2017-03-10 15:30:37 GMT <fwu> hi all!

2017-03-10 15:31:42 GMT <AFaust> yreg: Just running a quick debug - looks like the String is a duplicate because it is reparsed every time by the Ibatis DynamicSqlSource class

2017-03-10 15:32:03 GMT <yreg> AFaust, even more overhead

2017-03-10 15:32:41 GMT <AFaust> Well - they sort of have to do that since the original SQL in the Ibatis mapper files has all those conditional elements in it that need to be evaluated

2017-03-10 15:33:32 GMT <AFaust> So temporarily it has to be a separate String instance - but then there could be deduplication when a identical query has already been generated

2017-03-10 15:33:50 GMT <yreg> hmm, parsed as from xml, or from an inmomory mapper ?

2017-03-10 15:34:14 GMT <yreg> AFaust, What is the use of these caches anyway

2017-03-10 15:34:36 GMT <yreg> isn't the alfresco caching from the other side enough?

2017-03-10 15:35:33 GMT <AFaust> It is parsed again from the in-memory representation of the XML config

2017-03-10 15:35:52 GMT <AFaust> No - the Alfresco caching is not necessarily enough...

2017-03-10 15:36:53 GMT <AFaust> Some specific queries cannot be cached efficiently by Alfresco (due to complexity of invalidation / object mapping) and having a SQL tier cache can avoid repeating expensive queries

2017-03-10 15:37:16 GMT <AFaust> e.g. looking up child associations based on QName patterns

2017-03-10 15:38:53 GMT <AFaust> this loads the list of all child associations and then applies the regex filtering in process. It would not make sense for Alfresco to cache the result of that filtering because it is very unlikely the same pattern will be requested again, but having the entire list of child association results cache on ORM layer avoids another trip to the DB if no modification has occured since then...

2017-03-10 15:40:03 GMT <yreg> but this entire cache is bound to transaction and gets evicted on commit right ?

2017-03-10 15:40:44 GMT *** IanW2 is now known as IanW1

2017-03-10 15:40:45 GMT <AFaust> yes, because afterwards it might not be consistent anymore

2017-03-10 15:40:56 GMT <AFaust> due to changes in other transactions

2017-03-10 15:43:17 GMT <yreg> Then I find it too much memory cost for too little "possible" benefit ... at least in my case, and I really believe that I should have been given the possibility of at least limiting this cache's size !

2017-03-10 15:50:07 GMT <AFaust> Well - talk to the Ibatis devs about that

2017-03-10 15:59:18 GMT <AFaust> Even the newest version of ibatis does not allow configuration of the cache in the executor and does not seem to deduplicate SQL strings - even though that would be as easy as calling String.intern()

2017-03-10 16:05:03 GMT <yreg> AFaust, are doinf what I think you are doing ?

2017-03-10 16:05:10 GMT <yreg> <grin>

2017-03-10 16:05:23 GMT <AFaust> maybe

2017-03-10 16:05:57 GMT * AFaust applies the label "GitHub Stalker" to yreg

2017-03-10 16:10:36 GMT <yreg> hehe

2017-03-10 16:11:49 GMT <douglascrp> AFaust yreg good things, I bet

2017-03-10 16:13:53 GMT <yreg> douglascrp, we were discussing a "potential improvement" to iBatis caching, and few minutes later AFaust forks the project

2017-03-10 16:15:03 GMT <yreg> Good weekend everyone !

2017-03-10 16:26:48 GMT <douglascrp> ~later tell yreg now I understood :D

2017-03-10 16:26:48 GMT <alfbot> douglascrp: The operation succeeded.

2017-03-10 17:23:49 GMT <eswbitto> AFaust I downloaded the amp files for the support tools yesterday and I think they are "empty". Then I decided to read the directions. :D Anywho, so in order to create amp files I need to compile it using maven? Do I need to do anything besides graft it into my SDK? Do I need to create both a repo amp and share amp?

2017-03-10 17:24:45 GMT <AFaust> eswbitto: I just checked the AMP download file for the Share tier and it is

2017-03-10 17:24:51 GMT <AFaust> ~400 KiB

2017-03-10 17:24:51 GMT <alfbot> AFaust: Error: "400" is not a valid command.

2017-03-10 17:24:56 GMT <AFaust> ~400 KiB

2017-03-10 17:25:41 GMT <AFaust> You should not need to build the AMPs at all... Which version did you put in the dependencies for your project?

2017-03-10 17:26:59 GMT <eswbitto> I actually haven't gotten that far yet. I spent most of the night trying to update my SDK to the newer version so that it will work on 5.1

2017-03-10 17:27:49 GMT <eswbitto> I had another virtual machine that I have 5.1 running on for testing I tried the amps on. When I went to the support tools they were empty.

2017-03-10 17:28:22 GMT <eswbitto> So that tells me a missed a step in setting it up.

2017-03-10 17:34:40 GMT <AFaust> ~later tell yreg: There you go - https://github.com/AFaust/mybatis-3/commit/50b767a747f29efb53424a3878a07920538b8f7d

2017-03-10 17:34:40 GMT <alfbot> AFaust: The operation succeeded.

2017-03-10 18:20:02 GMT <eswbitto> AFaust how do I get all the support tool options that are shown here: http://www.zylk.net/en/web-2-0/blog/-/blogs/ootb-support-tools-addon-for-alfresco-community

2017-03-10 18:20:03 GMT <alfbot> Title: - OOTB Support Tools addon for Alfresco Community - zylk (at www.zylk.net)

2017-03-10 18:20:40 GMT <eswbitto> So far I only see 3. Dashboard Components, Log4j Settings and User Dashboards

2017-03-10 18:23:37 GMT <AFaust> you need to access the Repository Admin Console

2017-03-10 18:23:49 GMT <alfbot> yreg: Sent 1 hour and 57 minutes ago: <douglascrp> now I understood :D

2017-03-10 18:23:50 GMT <alfbot> yreg: Sent 49 minutes ago: <AFaust> There you go - https://github.com/AFaust/mybatis-3/commit/50b767a747f29efb53424a3878a07920538b8f7d

2017-03-10 18:23:52 GMT <AFaust> you are currently only looking at Share

2017-03-10 18:28:27 GMT <eswbitto> ah!

2017-03-10 18:28:32 GMT <eswbitto> AFaust thanks!

2017-03-10 20:35:30 GMT <yregaieg> ~later tell AFaust I can see in the config of the executor from the thread dump that caches should be disabled (cacheEnabled=false) Neverthless, it is taking 2.8Gb of my precious heap !

2017-03-10 20:35:30 GMT <alfbot> yregaieg: The operation succeeded.

2017-03-10 20:50:05 GMT <yregaieg> ~later tell AFaust s/thread dump/memory dump/

2017-03-10 20:50:05 GMT <alfbot> yregaieg: The operation succeeded.

End of Daily Log

The other logs are at http://esplins.org/hash_alfresco