cern-search-rest-api issueshttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues2022-01-14T11:47:05+01:00https://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/112Bug: Multiple entries in table ObjectVersion with is_head=True2022-01-14T11:47:05+01:00Carina AntunesBug: Multiple entries in table ObjectVersion with is_head=TruePossibly a race condition since it affects a very small percentage of files eg. 0.1%
Leads to invalid state in those specific docs in which updates are not possible and return a 500 error. Breaks in the code in the queries of type `one_...Possibly a race condition since it affects a very small percentage of files eg. 0.1%
Leads to invalid state in those specific docs in which updates are not possible and return a 500 error. Breaks in the code in the queries of type `one_or_none` on the table ObjectVersion (key, bucket_id, is_head=True)
https://cern-search-sentry.web.cern.ch/searchaas/edms-search-prod/issues/97112/?query=one_or_noneCarina AntunesCarina Antuneshttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/111Bump tika (log4j vulnerability)2022-01-11T19:15:19+01:00Carina AntunesBump tika (log4j vulnerability)Version 2.0.0 forward breaks metadata in unpack/all (From 1.27) - Bug report: https://issues.apache.org/jira/browse/TIKA-3624
Block: Release 2.2.1Version 2.0.0 forward breaks metadata in unpack/all (From 1.27) - Bug report: https://issues.apache.org/jira/browse/TIKA-3624
Block: Release 2.2.1Carina AntunesCarina Antuneshttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/110Fix sorting order2022-03-17T16:04:33+01:00Carina AntunesFix sorting orderCheck indico which is fixedCheck indico which is fixedhttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/108Investigate support for bulk ingestion2021-07-14T16:49:54+02:00Carina AntunesInvestigate support for bulk ingestionhttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/107Investigate how to modularize Client mappings and configs2021-07-14T16:45:08+02:00Carina AntunesInvestigate how to modularize Client mappings and configsSimilar to ES
https://gitlab.cern.ch/ai/it-puppet-hostgroup-it_es/-/blob/qa/data/hostgroup/it_es/search/pub7.yaml
or https://gitlab.cern.ch/it-elasticsearch-projectSimilar to ES
https://gitlab.cern.ch/ai/it-puppet-hostgroup-it_es/-/blob/qa/data/hostgroup/it_es/search/pub7.yaml
or https://gitlab.cern.ch/it-elasticsearch-projecthttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/106[ASR] Add new client config2021-10-25T14:16:23+02:00Carina Antunes[ASR] Add new client configNoor Afshan FathimaNoor Afshan Fathimahttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/104Integrate invenio-index-migrator2022-03-17T16:06:05+01:00Carina AntunesIntegrate invenio-index-migratorAllow migration of indexes with no downtime
https://github.com/inveniosoftware/invenio-index-migrator/blob/master/tests/conftest.py#L54Allow migration of indexes with no downtime
https://github.com/inveniosoftware/invenio-index-migrator/blob/master/tests/conftest.py#L54https://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/103Support post record/:id with external id2021-07-15T16:51:35+02:00Carina AntunesSupport post record/:id with external idCarina AntunesCarina Antuneshttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/102pin elasticsearch py2021-07-14T16:54:56+02:00Carina Antunespin elasticsearch pyDetails: https://mattermost.web.cern.ch/it-dep/pl/h86gj4o3oif8dpzi8ap4x8boeoDetails: https://mattermost.web.cern.ch/it-dep/pl/h86gj4o3oif8dpzi8ap4x8boeohttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/99Implement invenio-files-processor module with Tika processor2020-01-16T10:30:23+01:00Carina AntunesImplement invenio-files-processor module with Tika processorImplement invenio-files-processor module with Tika processor
https://github.com/inveniosoftware/invenio-files-processor/pull/11Implement invenio-files-processor module with Tika processor
https://github.com/inveniosoftware/invenio-files-processor/pull/11https://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/98Allow parsing from binary streams in tika python2020-01-16T10:28:56+01:00Carina AntunesAllow parsing from binary streams in tika pythonAllow parsing from binary streams
https://github.com/chrismattmann/tika-python/pull/269Allow parsing from binary streams
https://github.com/chrismattmann/tika-python/pull/269https://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/97[file-indexing] Add file processing queue + worker2020-01-16T09:40:50+01:00Carina Antunes[file-indexing] Add file processing queue + workerAdd file processing queue + Create celery worker to read from queue + Send file_uploaded to Queue
More info:
https://invenio-celery.readthedocs.io/en/latest/Add file processing queue + Create celery worker to read from queue + Send file_uploaded to Queue
More info:
https://invenio-celery.readthedocs.io/en/latest/Carina AntunesCarina Antuneshttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/96Investigate max capacity/ load test2019-12-05T15:25:30+01:00Carina AntunesInvestigate max capacity/ load testhttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/95Decision: File Indexing2020-01-20T09:42:33+01:00Carina AntunesDecision: File IndexingCarina AntunesCarina Antuneshttps://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/92Profile create record changes2019-11-25T10:20:49+01:00Carina AntunesProfile create record changesProfile create record changes due to https://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/issues/85.
If negative impacts is found make files with buckets configurable.Profile create record changes due to https://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/issues/85.
If negative impacts is found make files with buckets configurable.https://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/90Logging: Debug logs broken inside docker / oc2019-11-21T11:24:03+01:00Carina AntunesLogging: Debug logs broken inside docker / ocFlags don't seem to work. Investigate and fix.Flags don't seem to work. Investigate and fix.https://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/89test: ui/test_ui::test_view is failling2019-11-14T16:54:33+01:00Carina Antunestest: ui/test_ui::test_view is failling```
__________________________________________________________________________________________ test_view ___________________________________________________________________________________________
appctx = <Flask 'invenio'>, base_clien...```
__________________________________________________________________________________________ test_view ___________________________________________________________________________________________
appctx = <Flask 'invenio'>, base_client = <FlaskClient <Flask 'invenio'>>
@pytest.mark.unit
def test_view(appctx, base_client):
> resp = base_client.post("/account/settings/applications/", follow_redirects=True)
tests/ui/test_ui.py:17:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/werkzeug/test.py:1039: in post
return self.open(*args, **kw)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/testing.py:227: in open
follow_redirects=follow_redirects,
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/werkzeug/test.py:1017: in open
response, new_location, environ, buffered=buffered
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/werkzeug/test.py:948: in resolve_redirect
return self.open(builder, as_tuple=True, buffered=buffered)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/testing.py:227: in open
follow_redirects=follow_redirects,
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/werkzeug/test.py:993: in open
response = self.run_wsgi_app(environ.copy(), buffered=buffered)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/werkzeug/test.py:884: in run_wsgi_app
rv = run_wsgi_app(self.application, environ, buffered=buffered)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/werkzeug/test.py:1119: in run_wsgi_app
app_rv = app(environ, start_response)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py:2463: in __call__
return self.wsgi_app(environ, start_response)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py:2449: in wsgi_app
response = self.handle_exception(e)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py:1866: in handle_exception
reraise(exc_type, exc_value, tb)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/_compat.py:39: in reraise
raise value
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py:2446: in wsgi_app
response = self.full_dispatch_request()
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py:1951: in full_dispatch_request
rv = self.handle_user_exception(e)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py:1820: in handle_user_exception
reraise(exc_type, exc_value, tb)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/_compat.py:39: in reraise
raise value
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py:1949: in full_dispatch_request
rv = self.dispatch_request()
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py:1935: in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask_security/decorators.py:230: in wrapper
return f(*args, **kwargs)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask_security/views.py:89: in login
**_ctx('login'))
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask_security/core.py:528: in render_template
return render_template(*args, **kwargs)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/templating.py:140: in render_template
ctx.app,
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/templating.py:120: in _render
rv = template.render(context)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/jinja2/asyncsupport.py:76: in render
return original_render(self, *args, **kwargs)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/jinja2/environment.py:1008: in render
return self.environment.handle_exception(exc_info, True)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/jinja2/environment.py:780: in handle_exception
reraise(exc_type, exc_value, tb)
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/jinja2/_compat.py:37: in reraise
raise value.with_traceback(tb)
/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_oauthclient/templates/invenio_oauthclient/login_user.html:11: in top-level template code
{%- from "invenio_oauthclient/_macros.html" import oauth_button %}
/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_accounts/templates/invenio_accounts/login_user.html:11: in top-level template code
{% from "invenio_accounts/_macros.html" import render_field, form_errors %}
/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_theme/templates/invenio_theme/page_cover.html:13: in top-level template code
{%- set body_css_classes=['cover-page'] %}
/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_theme/templates/invenio_theme/page.html:12: in top-level template code
{%- block head %}
/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_theme/templates/invenio_theme/page.html:47: in block "head"
{%- block css %}
/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_theme/templates/invenio_theme/page.html:48: in block "css"
{{ webpack['theme.css']}}
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/jinja2/environment.py:411: in getitem
return obj[argument]
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/werkzeug/local.py:378: in <lambda>
__getitem__ = lambda x, i: x._get_current_object()[i]
/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/werkzeug/local.py:307: in _get_current_object
return self.__local()
/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flask_webpackext/proxies.py:22: in <lambda>
lambda: current_app.extensions['flask-webpackext'].manifest)
/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flask_webpackext/ext.py:88: in manifest
join(self.app.static_folder, path))
/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flask_webpackext/manifest.py:66: in load
super(JinjaManifestLoader, self).load(filepath)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <flask_webpackext.manifest.JinjaManifestLoader object at 0x7f270187df98>, filepath = '/opt/rh/rh-python36/root/usr/var/instance/static/dist/manifest.json'
def load(self, filepath):
"""Load a manifest from a file."""
> with open(filepath) as fp:
E FileNotFoundError: [Errno 2] No such file or directory: '/opt/rh/rh-python36/root/usr/var/instance/static/dist/manifest.json'
/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pywebpack/manifests.py:212: FileNotFoundError
```https://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/87global: expose health status2019-11-06T10:51:34+01:00Pablo Panero Vazquezglobal: expose health statusExpose health status to the outside. This would allow clients to check if their search instance is up and running.
Need to aggregate the `/health/{es,db,uwsig}` (web app) + `/ping` (nginx) endpoint responses.Expose health status to the outside. This would allow clients to check if their search instance is up and running.
Need to aggregate the `/health/{es,db,uwsig}` (web app) + `/ping` (nginx) endpoint responses.https://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/84global: linux incompatibilities2020-06-11T16:11:40+02:00Pablo Panero Vazquezglobal: linux incompatibilities* Docker-compose version is set to 3.7, the dockerfile is compatible with at leas 2.3 lower.
- Shall we provide support for 2.x? (The whole Invenio framework still users 2.
* `shell` might not be available in all systems.
- Requi...* Docker-compose version is set to 3.7, the dockerfile is compatible with at leas 2.3 lower.
- Shall we provide support for 2.x? (The whole Invenio framework still users 2.
* `shell` might not be available in all systems.
- Require it or avoid using it
- `/scripts/pipenv/requirements.sh` might crash if the command is run in another shell since `[[` is not always available (see [here](https://stackoverflow.com/questions/12230690/string-comparison-in-bash-not-found/12230723))https://gitlab.cern.ch/webservices/cern-search/cern-search-rest-api/-/issues/81Lock all openshift dependencies2019-10-18T12:17:12+02:00Carina AntunesLock all openshift dependenciesWithout pinning images will be pulled using the very latest version at build time.
To avoid an unstable app state, avoid things breaking with new dependencies releases and avoid confusion on supported environments lock all images versio...Without pinning images will be pulled using the very latest version at build time.
To avoid an unstable app state, avoid things breaking with new dependencies releases and avoid confusion on supported environments lock all images versions.
Current unpined versions:[nginx](https://gitlab.cern.ch/webservices/cern-search/cern-search-openshift/blob/master/rest-api/application.yaml#L64) and [redis](https://gitlab.cern.ch/webservices/cern-search/cern-search-openshift/blob/master/rest-api/services.yaml#L42).
Update docs with the images versions supported.
Check if any other exists. (web-ui)