Commit fb13df94 authored by Domenico Giordano's avatar Domenico Giordano
Browse files

Update control_room/airflow-compose/dags/README.md,...

Update control_room/airflow-compose/dags/README.md, control_room/airflow-compose/config_files/publish_fluentd_always_on/batch_etl_test.yaml, control_room/airflow-compose/config_files/README.md, control_room/airflow-compose/docker-swarm/swarm-docker-compose.yml, control_room/airflow-compose/docker-swarm/swarm_step2_start_swarm.sh, control_room/airflow-compose/docker-swarm/swarm_step_1_pass_credentials_to_cluster.sh, control_room/airflow-compose/README.md, control_room/README.md, control_room/airflow-compose/config_files/always_on/analysis_eos.yaml, control_room/airflow-compose/config_files/always_on/etl_test_eos.yaml, control_room/airflow-compose/config_files/always_on/etl_train_eos.yaml, control_room/airflow-compose/config_files/benchmark/essential_metrics/batch_etl_test.yaml, control_room/airflow-compose/config_files/benchmark/essential_metrics/batch_etl_train.yaml, control_room/airflow-compose/config_files/benchmark/essential_metrics/batch_experiment.yaml, control_room/airflow-compose/config_files/benchmark/essential_metrics/shared_etl_test.yaml, control_room/airflow-compose/config_files/benchmark/essential_metrics/shared_etl_train.yaml, control_room/airflow-compose/config_files/benchmark/essential_metrics/shared_experiment.yaml, control_room/airflow-compose/config_files/benchmark/improve_metrics/batch_etl_test.yaml, control_room/airflow-compose/config_files/benchmark/improve_metrics/batch_etl_train.yaml, control_room/airflow-compose/config_files/benchmark/improve_metrics/shared_etl_test.yaml, control_room/airflow-compose/config_files/benchmark/improve_metrics/shared_etl_train.yaml, control_room/airflow-compose/config_files/benchmark/noise_metrics/batch_etl_test.yaml, control_room/airflow-compose/config_files/benchmark/noise_metrics/batch_etl_train.yaml, control_room/airflow-compose/config_files/benchmark/noise_metrics/shared_etl_test.yaml, control_room/airflow-compose/config_files/benchmark/noise_metrics/shared_etl_train.yaml, control_room/airflow-compose/config_files/benchmark/batch_etl_test.yaml, control_room/airflow-compose/config_files/benchmark/batch_etl_train.yaml, control_room/airflow-compose/config_files/benchmark/batch_experiment.yaml, control_room/airflow-compose/config_files/benchmark/etl_test_shared.yaml, control_room/airflow-compose/config_files/benchmark/etl_train_shared.yaml, control_room/airflow-compose/config_files/benchmark/experiment_shared.yaml, control_room/airflow-compose/config_files/benchmark/experiment_shared_lof.yaml, control_room/airflow-compose/config_files/demo/analysis_eos.yaml, control_room/airflow-compose/config_files/demo/etl_test.yaml, control_room/airflow-compose/config_files/demo/etl_test_eos.yaml, control_room/airflow-compose/config_files/demo/etl_train.yaml, control_room/airflow-compose/config_files/demo/etl_train_eos.yaml, control_room/airflow-compose/config_files/demo/template_analysis.yaml, control_room/airflow-compose/config_files/publish_fluentd/batch_etl_test.yaml, control_room/airflow-compose/config_files/publish_fluentd/batch_etl_train.yaml, control_room/airflow-compose/config_files/publish_fluentd/batch_experiment.yaml, control_room/airflow-compose/config_files/publish_fluentd_always_on/batch_etl_train.yaml, control_room/airflow-compose/config_files/publish_fluentd_always_on/batch_experiment.yaml files
Deleted control_room/airflow-compose/troubleshooting.md
parent de23bf17
......@@ -56,6 +56,8 @@ curl https://gitlab.cern.ch/cloud-infrastructure/data-analytics/-/raw/$branch/co
install_all
```
Follow then the instructions printed by the install_AD.sh script to finalise the setup
3. Start the docker compose of the Airflow-based Anomaly Detection System with the following command:
```
sudo -u airflow /opt/ad_system/control_ad_system/start_ad_system.sh
......@@ -66,7 +68,8 @@ NB: the script `/opt/ad_system/control_ad_system/start_ad_system.sh` can also be
**Congratulation!** You just complete the full installation of your Anomaly Detection System.
## Getting started with your first Anomaly Detection DAG
## Getting started with your first Anomaly Detection DAG
**W.I.P. : should it be moved in another README?**
Now that Airflow is up and running we can test the Anomaly Detection System and
its algorithms on a demo scenario.
......
......@@ -31,13 +31,7 @@ This is the standard method. The instructions are in the main [README.md](../REA
## Docker Swarm
This is still W.I.P.
In case of a docker swarm, run to start
```
./swarm_step1_pass_credentials_to_cluster.sh
./swarm_step2_start_swarm.sh
```
In case of a docker swarm, run to start look into the script in folder [docker-swarm](./docker-swarm)
## Kubernetes
......@@ -49,3 +43,18 @@ Documentation can be found at
- https://airflow.apache.org/docs/stable/kubernetes.html
## Troubleshooting
This section collects some expedients applied in case of issues.
Not necessary all of them will be always valid. This section needs a recurrent review.
### docker-compose network not starting
Creating network "airflow-compose_default" with the default driver
ERROR: Failed to Setup IP tables: Unable to enable SKIP DNAT rule: (iptables failed: iptables --wait -t nat -I DOCKER -i br-37e312e2428d -j RETURN: iptables: No chain/target/match by that name.
(exit status 1))
```
systemctl restart docker
systemctl restart iptables
iptables -t filter -N DOCKER
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment