NXCALS-1765 gobblin cleanup

#Merge request template @acc-logging-team - please have a look for approval.

Please take an especially careful look on my ansible changes. I really wanted to work within the given structure, but you never know...

features

• every etl instance, prior to launching, now will check if it has directories to clean (param clean.directories)
• if so, it will recursively remove all the empty directories starting from the given roots (potentially removing even the root itself)

minor

• utility class DirectoryIterator, which discovers all the subdirectories starting from a given root, skipping if the content triggers the predicate (responding to additions and removals)
• utility class HdfsCleaner, which removes all the directories given by an iterator, measures the removal time and size

Closes NXCALS-1765

added 1 commit

• 53cfbf86 - NXCALS-1765 added G1 to pro, test, testbed and staging

Compare with previous version

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported 11 issues

• 11 major

Note: The following issues were found on lines that were not modified in the commit. Because these issues can't be reported as line comments, they are summarized here:

1. Invoke method(s) only conditionally. Use the built-in formatting to construct this argument.
2. Format specifiers should be used instead of string concatenation.
3. Use the built-in formatting to construct this argument.
4. Format specifiers should be used instead of string concatenation.
5. Use the built-in formatting to construct this argument.
6. Format specifiers should be used instead of string concatenation.
7. Define and throw a dedicated exception instead of using a generic one.
8. Rename "next" which hides the field declared at line 26.
9. Define and throw a dedicated exception instead of using a generic one.
10. Define and throw a dedicated exception instead of using a generic one.
11. Rename "next" which hides the field declared at line 26.

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported 11 issues

• 11 major

Note: The following issues were found on lines that were not modified in the commit. Because these issues can't be reported as line comments, they are summarized here:

1. Invoke method(s) only conditionally. Use the built-in formatting to construct this argument.
2. Format specifiers should be used instead of string concatenation.
3. Use the built-in formatting to construct this argument.
4. Format specifiers should be used instead of string concatenation.
5. Use the built-in formatting to construct this argument.
6. Format specifiers should be used instead of string concatenation.
7. Define and throw a dedicated exception instead of using a generic one.
8. Rename "next" which hides the field declared at line 26.
9. Define and throw a dedicated exception instead of using a generic one.
10. Define and throw a dedicated exception instead of using a generic one.
11. Rename "next" which hides the field declared at line 26.

added 1 commit

• 3cdee4b6 - NXCALS-1765 sonar fixes

Compare with previous version

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.

SonarQube analysis reported no issues.