Workflow for gateware development on the DIOT platform

Starting from #53 (comment 7819958), I'm opening this issue to discuss the gateware development workflow. For context, at the moment I'm trying to add some IPs to the system board, to carry out some tests that may eventually find their way into production.

Unfortunately, I was not really able to work on the existing build system. For ease of my development, I wanted to just make changes on the block design level. My undestanding is that the current build system was not really developed with an iteraive usage in mind, but rather for a "single shot".

To aid my development, I did some changes to the current TCL script for building the gateware, in an attempt to make the build more modular (branch: boot-image/-/tree/buildsystem-proposal). The criticalities I tried to address are:

1. The developer cannot run make all twice without calling make clean, this is not something I would expect;
2. Calling make clean would delete intermediate build products but the whole project, including changes to the project files. This is something I would expect from a make distclean but not from a make clean;
3. There is no way to simply create the project and populate the block design.

As a consequence, developing locally on the Vivado project was not possible. By splitting the build_gw.tcl file into functions and passing a command to it (which was already done really, just in a less structured way) and by defining more granular targets in the makefile, the buildsystem allows to first create the project and it's files, and later to issue the command to generate the bitstream and the required outputs.

While this is a suitable strategy to mitigate the development difficulties I have now, this also raised some questions about the general development workflow. How are developers expected to interact with this repository? If we need to develop some gateware "on top of the platform" on the SB, the option I see right now is to fork this repository and to edit the tcl scripts to include the custom gateware, but this raises the question on how to easily keep in sync with the upstream repo, in case the platform gets updated. What is your view on this? Bear in mind that for now I have only worked with the gateware side of this project, so my view could be partial and not accurate.

One last question: Is there a particular reason why the whole build system was created as many separate makefiles instead of a single one? I could see just a single "make all" call to make the BOOT.BIN. In particular, a build system that is aware of dependencies and know which parts need to be recompiled following some sources update would be great.

Thank you very much for all your work and support! 😄