python - Celery add tasks to queue when ready -

-

I am new to celery I am trying to do a distributed function with celery

Let's say that I have any job in my work file task.py : 

  @ celery.task dff gener_sorts (params, meta_data_dict): '' 'This will generate some type' '' pass   
 and I am doing the following as part of some distributed resources:  
  taskset = TaskSet (tasks.generate_sorts.subtask (args = (Params, meta_data_dict)) Meta_Data_Disk Chunk_generator for print) Print "Task Disclosure" taskset_result = taskset.apply_async () Print "Waiting for results" Results = WorkComedia_Salt. Join_native () Print "Result:" Process # Results   
 Now the chunk_generator basically fetches a generator pattern which goes to the database and some metadata. My problem is that these work gets deposited before the work queue is already sent. In fact, before generating in the queue, my generator takes about 30 minutes to fetch all meta data. I know how it works for  TaskSet  i  TaskSet  I'm looking for some alternative, that is, I will be able to do the equivalent in the distributed way below. 
 
  pool.imap_unordered (gener_sorts, chunk_generator)   
 The above gener_sots will display as soon as a generator generates results. In other words, there is anything optional for  TaskSet , where I can get a job from the generator for the first time, before the generator can bring everything to the generator instead of waiting to get it, before That I can start some work.   
 
 
 You try to start them immediately and try to add examples as a result of  asyncResult  Should: A:  
  result result from result of sales_set = ResultSet () chunk_generator in the meta_data_choice: select the job immediately in the # queue result = task.generate_sorts.delay (params, meta_data_dict) result_set Results (results) Print "Waiting for result" result = result_set.join_native () print "results:" take action on # result

Comments

Post a Comment

Popular posts from this blog

php - PDO bindParam() fatal error -

-
I am trying to learn PDO, now I have written this small code, but it gives me a serious error: Fatal error: Call a member function at a non-object dot () ... $ con = new mysqli ("127.0.0.1", "root", "", "csvdangercheck"); $ Query = $ con- & gt; ("Id,` var1`, `var2`) values ​​(" id,: var1,: var2) ";); $ query- & gt; bypass pattern (': id', $ id); $ query- & gt; BindParam (': var1', $ VAL1); $ Query- & gt; bindParam (': var2', $ val2); $ Query-> Executed (); I Trying to use print_r ($ con-> errorInfo ()) but Fatal error: Undefined method mysqli :: errorInfo ( ) ... can someone tell me what am I missing? Flaffeh said, you have PDO Missing Mysqy, try it: $ con = new PDO ('mysql: host = 127.0.0.1; dbname = csvdangercheck', 'root', '');
Read more

php - How can I cram 6+31 numeric characters into 22 alphanumeric characters? -

-
I have a 6 digit number and a 31-digit number (such as "234536" and "201103231043330478311223582826") that I The API needs to be crawled in the same 22-character alphanumeric field that uses PHP. I tried to convert each one to 32 (using custom code could not handle the base_convert () large number) and to connect with a single-character delimiter, but It only gives me 26 characters This is a restore API, so the letters should be URI-protected. I would like to do a database table cross with two references without reference to another reference value, if possible, any suggestions? Instead use a radix of 62. This will give you 22 characters for the top, 3.35 for the pre and 17.3 for the next. & gt; & Gt; & Gt; Math.log (10 ** 6) /math.log (62) 3.3474826039165504 & gt; & Gt; & Gt; Math.log (10 ** 31) /math.log (62) 17.295326786902177
Read more

logging - How can I log both the Request.InputStream and Response.OutputStream traffic in my ASP.NET MVC3 Application for specific Actions? -

-
For a specific set of tasks, I need to log in to the incoming request inputstream as well as outgoing response. is. I am using an ActionFilterAttribute for this and overriding the OnActionExecuted and OnResultExecuted methods. So I'm starting with this idea ... Public category ActionLoggerAttribute: ActionFilterArrbit {Public Override Zero On-Offed Execution (ActionActioned Contact Filter Filter) {base.OnActionAxecuted (FilterContext); HttpRequestBase request = filterContext.HttpContext.Request; // TODO: Log the request. InputStream} Public Override Zero OnResultExecuted (ResultExecutedContext FilterContext) {base.OnResultExecuted (filterContext); HTTPRPSPointBase Response = FilterContacts Hpptex response; // TODO: Log on Response.OutputStream}} Ideally I'll hook it up with enterprise library logging because I'm already using it for error logging. Am I reaching the input and output stream at the right time? Can I easily use the Enterprise Library to log i...
Read more