Add codimd pipeline

Supports oais-web#59 (closed)

When the user selects codimd as a source, no matter what the value of the query, search by id is disabled and clicks on the submit button then a list of all his codimd notes will be displayed on the harvest page. You can use the codimd history/ api call as it is used in bagit-create: https://gitlab.cern.ch/digitalmemory/bagit-create/-/blob/develop/examples/codimd_history.py and return a list of all the notes based on the codimd history.

Steps:

• At oais/sources/source.py, add codimd as a source
• Create a new codimd.py file in source folder.
• You should have 2 functions search and search_by_id. The first one should return all the notes of a user and the second one should return a specific note with the id passed on the query field.
• The results should be returned in a proper format:

{"source_url": codimd url,
"recid": codimd id,
"title": title of the note,
"authors": name of the user,
"source": "codimd"}

(you can use indico.py and https://gitlab.cern.ch/digitalmemory/bagit-create/-/blob/develop/examples/codimd_history.py for help)

• You can try to apply pagination for the search function so based on the size and page variables, you car return the respective results. (ex. if page=1, size=10 you return results (0:9), if page=3, size=10 you return (20:29))
• For paginated results, the response from the server should have the following format:

{
   "total_num_hits": SIZE OF RESULTS, 
   "results": {
      [
         {
            "source_url": codimd url,
            "recid": codimd id,
            "title": title of the note,
            "authors": name of the user,
            "source": "codimd"}
         },{
            "source_url": codimd url,
            ...
      ]
   }
}

• for the search_by_id you must return 1 result if exists with the same format or none if this id does not exist

In this case the response from the server should have this format:

{"result": {
   "source_url": codimd url,
   "recid": codimd id,
   "title": title of the note,
   "authors": name of the user,
   "source": "codimd"}
}}

Run bagit-create

• When the staged record is created and approved, then process function in tasks.py is invoked. You have to make sure that codimd_api_token is passed when calling the bic.process function

Hints

• Open the developer console and at the network tab check the api calls and the request/response data structure
• Check oais/sources/indico.py for more insight as the pipeline which must be created is very similar to the one of indico