Quantcast
Channel: New Topics on Talend Community Forum
Viewing all articles
Browse latest Browse all 2816

Question on variable scope: subjobs, joblets, tRunJob, tJavaRow, tJava - by indy2005

$
0
0

Hi, I would like some information on variable scope when working within a subjob, working across multiple subjobs in the same job, when executing a joblet vs a tRunJob.   How are variables shared, context shared and are globalMaps shared or not?  I am also interested in how these are impacted when you parallelize a flow, for example using round robin allocation to threads from an input component.   I am trying to create a complex, multi level mongoDB document, and want to speed it up through parallelization, but I have a high dependency on storing documents I am creating in the globalMap, so I can carry on down the flow, execute more SQL parameterised by globalMap variables and build an embedded document. It works, but it is slow...so I want to make the first tOracleInput parallelized, so each row goes out to a different thread....but then my globalMaps will be trampling all over each other. I tried writing things into the globalMap using a combination of the threadID and the object key name, using globalMap.get("tCollector1_THREAD_ID"), but I don't understand how this gets populated...surely if its in the globalMap, the threads are trampling all over this value all the time also? Perhaps I should create joblets or subjobs to make sure I have separate scopes, and pass things by reference using contexts?  Is there any advice or documentation on this please? Regards i


Viewing all articles
Browse latest Browse all 2816

Trending Articles