Processing a simple workflow in Python -

-

I am working on a code that takes the dataset and runs some algorithms on it.

User uploads a dataset, and then selects the algorithm to run on this dataset and creates a workflow like this: 

  workflow = {0: {' Dataset ':' some dataset ', 1: {' algorithm1 ': "parameter"}, 2: {' algorithm2 ': "parameter"}, 3: {' algorithm 3 ': "parameter"}}

which means that I will run the workflow [0] as my dataset, and I will run it as algorithm1 . After that, I will take the result and I will run the algorithm2 on this result like my new dataset. And I will take new results and run it on algorithm3 This happens to this last item and there is no limit to this workflow.

I am writing it in Python. Can you recommend some strategy about processing this workflow?

You want to run a pipeline on some datasets. It looks like less operation (fold in some languages). 

  result = less (lambda data, (anonym, p): algo_by_name (anonymous) (p, data), workflow)

This workflow considers (text-oriented so that you can load it with YAML / JSON): 

  workflow = ['data', ('algo0', {}), '(Algo1 ', {' Param ': value}), ...]

And that looks like your algorithm: 

  def algo0 (p , Data): ... return_output_filename

takes algo_by_name and gives you an algo function; For example: 

  def algo_by_name (name): return {'algo0': algo0, 'algo1': algo1,} [name]

( Edit old: If you want a framework to write a pipeline, you can use it as it is like making an instrument, but with progress support and beautiful flow chart.)

< / Html>

Comments

Post a Comment

Popular posts from this blog

oracle - The fastest way to check if some records in a database table? -

-
I have a big table to work with. I have to see that there are some records whose parent_id is equal to my passing value. Currently I do this by implementing "my_table using select count (*) where parent_id =: id"; If the result> 0, it means that they exist. Because this is a very large table, and I do not care what the actual number of records present, I just want to know if it exists, so I think count (*) Bit is disabled. How do I apply this requirement most quickly? I'm using Oracle 10. # Tips for Hibernation & amp; Tricks It suggests writing in such a way: Integer number = (integer) session .createQuery ("Select Count (*) to ...."). UniqueResult (); I do not know what is the magic of the unique rule? Why does it fast? To compare, select "1 from mytable where parent_id = passId and rowrum", which is more efficient? If you are not interested in the number of records, then there is an EXISTS query one: Select 'y'
Read more

php - multilevel menu with multilevel array -

-
Just ask someone to solve this problem? I want to create a multilevel menu I can not find the right solution generated with multilevel array, it always gets an array_push error My study came from the original idea But still my Can not match the need. This is my MySQL [code] ----- --------------------- --------------- | ID | PARENT_ID | Name | Link | Seq | 1 | 0 | Dashboad | Dashboard / | 1 | 2 | 0 | Menu 1 | Menu 1 / | 2 | 3 | 0 | Menu 2 | Menu 2 / | 3 | 4 | 0 | Menu 3 | Menu 3 / | 4 | 5 | 2 | Joint Add / | 1 | 6 | 3 | Joint Add / | 1 | 7 | 2 | Edit | Edit / | 2 | 8 | 4 | Joint Add / | 1 ------------------------------------------ [/ Code] I want to do something like this in my array array ('dashboard' = & gt; array ('id' = & gt; '' 'Dame' = & gt; 'Dashboard', 'Title' = & gt; 'Dashboard', 'MatchMin' = & gt; Base_URL (). 'Dashboard /', 'Active' => = & Gt; NULL), &
Read more

jQuery UI: Datepicker month format -

-
I am trying to change the format of the month that shows in the popup (not in the date that the picker actually I do not have any choice anywhere to do this, do I like to use a sub-string () or something else? What I normally display is Trying to change: Class = "ui-datepicker-title"> January & lt; / span & gt; ... & lt; / div & gt; Div>
Read more