python - How can I set up Celery to call a custom initialization function before running my tasks? -

-

I have a dynamic project and I try to use celery to submit jobs for background processing () I am here. The celery integrates well with jigs and I am capable of submitting my custom tasks and getting the results back.

The only problem is that I can not find the sensible method of initializing customization in the daemon process. I have to call an expensive task that loads a lot of memory before processing the processing, and I I am not able to call that function every time.

Does anybody have this problem before? Any idea how to work it without modifying the cellular source code?

Thanks

You can write a custom loader or use a signal .

The loader has the on_task_init method, which is said to be executed when the task is executed, and the which is called the cellular + serial bit main process.

Using signals is the easiest, there are available signals:

0.8.x:

  • Task_progress (task_id, Work, args, quad)

    The worker has been dispatched to perform the work (or locally if applies / or if CELERY_ALWAYS_EAGER is set).

  • task_postrun (task_id, task, args, kwargs, retval) after one task has been executed in the above terms.

  • task_cent (task_id, task, args, kwargs, eta, taskset)

    When a task is applied (long Not enough till the time) Additional signals available in 0.95 (Current Master branch on Gitball):

    • Worker_init ()

      When the cellarid has started (before the work begins, so if there is a system supporting fork , any memory change child Will be copied to the worker process).

    • worker_ready ()

      Selereed is called to be able to get work.

    • worker_shutdown ()

      When the cell is off, the call is made.

    Here is an example of the first run in a Work for the first time: 

      from celery.task import celery.registr task imports of Y Import Kren_sath_purw_prerkel_tel = {} class Paursoftio (task): def run (self-X): If _precalc_table X: return _precalc_table [x] else: return x ** 2 taskss.register (PowersOfTwo) Def _precalc_numbers ( ** kwargs): If not _precalc_table: # it is empty, so I was not born yet (1024) to the limit: _precalc_table [i] = i ** 2 # from the logic of Is required to use a registered instance. Task_prerun.connect (_precalc_numbers, from = function [PowerOfTwo.name])

    If you want to run the function for all functions, just from Leave the argument.


    • Comments

    Post a Comment

    Popular posts from this blog

    oracle - The fastest way to check if some records in a database table? -

    -
    I have a big table to work with. I have to see that there are some records whose parent_id is equal to my passing value. Currently I do this by implementing "my_table using select count (*) where parent_id =: id"; If the result> 0, it means that they exist. Because this is a very large table, and I do not care what the actual number of records present, I just want to know if it exists, so I think count (*) Bit is disabled. How do I apply this requirement most quickly? I'm using Oracle 10. # Tips for Hibernation & amp; Tricks It suggests writing in such a way: Integer number = (integer) session .createQuery ("Select Count (*) to ...."). UniqueResult (); I do not know what is the magic of the unique rule? Why does it fast? To compare, select "1 from mytable where parent_id = passId and rowrum", which is more efficient? If you are not interested in the number of records, then there is an EXISTS query one: Select 'y'
    Read more

    php - multilevel menu with multilevel array -

    -
    Just ask someone to solve this problem? I want to create a multilevel menu I can not find the right solution generated with multilevel array, it always gets an array_push error My study came from the original idea But still my Can not match the need. This is my MySQL [code] ----- --------------------- --------------- | ID | PARENT_ID | Name | Link | Seq | 1 | 0 | Dashboad | Dashboard / | 1 | 2 | 0 | Menu 1 | Menu 1 / | 2 | 3 | 0 | Menu 2 | Menu 2 / | 3 | 4 | 0 | Menu 3 | Menu 3 / | 4 | 5 | 2 | Joint Add / | 1 | 6 | 3 | Joint Add / | 1 | 7 | 2 | Edit | Edit / | 2 | 8 | 4 | Joint Add / | 1 ------------------------------------------ [/ Code] I want to do something like this in my array array ('dashboard' = & gt; array ('id' = & gt; '' 'Dame' = & gt; 'Dashboard', 'Title' = & gt; 'Dashboard', 'MatchMin' = & gt; Base_URL (). 'Dashboard /', 'Active' => = & Gt; NULL), &
    Read more

    jQuery UI: Datepicker month format -

    -
    I am trying to change the format of the month that shows in the popup (not in the date that the picker actually I do not have any choice anywhere to do this, do I like to use a sub-string () or something else? What I normally display is Trying to change: Class = "ui-datepicker-title"> January & lt; / span & gt; ... & lt; / div & gt; Div>
    Read more