Resolve "Rendered yaml is too long"

As in, https://github.com/steemit/sbds/issues/63, modifies the rendered_yaml to accept longer objects (up to 16MB with MEDIUMTEXT).

Created to solve #41 (closed)

added Bug WP2 - Analysis Productions labels

restored source branch 41-rendered-yaml-is-too-long

added 1 commit

• 807607e5 - Suggested fix for too long rendered yamls.

Compare with previous version

unmarked as a Work In Progress

changed the description

@mmulder, the pipeline is red ...

For reference - a test for this is being put forward at lbap-testing!6 (closed).

assigned to @cburr

Thanks for pointing that out @erodrigu; I had indeed not seen that. I am retrying just to see what is causing it.

On the side of black it is not clear to me what should be fixed;

On the side of basic_tests it seems to be some kind of permissions issue?

@cburr was planning to have a look at this suggested fix this week (but I think he is quite busy this week); perhaps he can comment on what he thinks might be going wrong?

Black is a pretty now standard formatting tool from the PSF. You can pip install it locally and run it for your file. You can then commit any changes that Black suggests. In my projects outside LHCb we have it as a pre-commit as part of the CI, and that helps. Maybe some better integration is needed ...

added 1 commit

• c39e2ee0 - Proper inclusion of with_variant

Compare with previous version

added 1 commit

• d2f7f4c7 - Reformatted w black

Compare with previous version

Thanks for that, I have reformatted with black and (hopefully) included with_variant properly now.

The failure in basic_tests does not seem to be only on permissions:

/opt/conda/envs/analysis-productions/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:737: in __getattr__
    replace_context=err,
E   AttributeError: Neither 'Column' object nor 'Comparator' object has an attribute 'with_variant'
No S3 credentials specified, disabling

It could well be that the sqlalchemy version used is too old? I have no knowledge of this package at all, though ...

added 1 commit

• 65e8d838 - Fix by actually looking at example

Compare with previous version

added 1 commit

• 273c538c - Reformatted w black

Compare with previous version

added 1 commit

• 4e97917a - Typo

Compare with previous version

resolved all threads

Okay, I resolved the previous issues; now some latter tests are failing and it is quite unclear to me why. It seems like there are issues with saving the jobs. Perhaps @cburr knows what is happening?

added 8 commits

• 4e97917a...bc6a3628 - 7 commits from branch master
• c09569af - Merge branch 'master' into 41-rendered-yaml-is-too-long

Compare with previous version

added 1 commit

• f0809026 - Also use MEDIUMTEXT for regular yaml

Compare with previous version

resolved all threads

approved this merge request

mentioned in commit c96192a0

merged

mentioned in merge request lbap-testing!6 (closed)