The problem is that (currently) Gaudi based projects set those variables (and in particular CMAKE_BUILD_TYPE) through the call to gaudi_project, which in Allen happens too late.
After gaudi/Gaudi!986 (merged), the CMAKE_BUILD_TYPE will be set by the CMAKE_TOOLCHAIN_FILE, but until then it would be better that Allen does not set it for NOT STANDALONE builds.
This issue is more subtle than it seems in my opinion.
We should identify the flags that both Gaudi and Allen use, and on those, adopt whatever convention Gaudi has decided on (ie. release, relwithdebinfo, debug flags).
However, there may be specific compile flag requirements from a specific host / device target architecture. If we assume only Allen will target architectures other than x86-64 and do heterogeneous computing, these don't need to be extended to any other projects.
Some examples that come to mind:
x86-64, the different ppc architectures, the different arm architectures, and specifically aarch64, have all different optimization options, and should have all different defaults.
Every different target device architecture, some supported by CMake, some not yet, have their set of parameters with defaults. For instance, for CUDA:
Some of these are going to change (CMAKE_CUDA_STANDARD will become 17 soon), some others have specific flags (-G is needed to generate debug symbols in CUDA). HIP has its own specific ones as well. CUDACLANG is not supported by cmake yet, but we have a functional compilation configuration for it.
Not all host compilers supported upstream will be supported by [insert language extension]. For instance, CUDA 11 (recently released) supports up to GCC-9 and Clang-9. It is likely that a future gcc-10 release will require a new compiler option that Allen will have to set off, and Allen will have to be compiled with a different host compiler than the rest of the codebase.
All in all, I think for now we can identify the common parts and propagate the Gaudi flags where needed. But it looks to me like an issue that we should keep actively tracking.
I agree that Allen has more complex requirements than other projects we have had so far, so its inevitable things will be more complicated with it. However I think you have it right, in that we should identify the basic build options that are set by Gaudi, and Allen should then respect these, as far as it can, and only modify/append when needed for specific reasons.
A few other comments
Its not necessarily correct only Allen has to target non x86_64. There is an ongoing effort/wish to get the whole stack building for Arm64, for instance.
gcc10/clang10 might be an issue sooner than you think. Both are in fact already the most recent production releases for each compiler, and I believe we will be starting to support them in an LCG update 'soon'. So if Allen is going to have a problem with this a solution will need to be found.
I risk to sound like a broken record, but I repeat that we should decouple device code (including CPU as a possible device target), from the code to interface with Gaudi applications. This split is going to address nicely:
different standard between CUDA and C++
different licenses for device (Apache-2) and the interfacing code (GPLv3)
integration with the build system (only the interface code has to blend into Gaudi, while the device code can be built as an external library)
It would not even be the first case for such a split: we already have GitCondDB and Detector that are following that path.
In any case, for sake of a quick workaround to the compelling problem, I suggest we just fix the CMAKE_BUILD_TYPE inconsitency. In a second phase we can address the remaining inconsistencies (that are going to be less of a problem if we separate the concerns).
In Allen, the separation can be made between the stream library (Apache-2), and the interfacing code (GPLv3).
The Stream library could be compiled as an external library. The Stream library contains a sequence with all algorithms of that sequence. For that, both device and host compilers are needed, since the sequence requires both host and device algorithms, and the steering code is actually compiled in the host (which is faster).
So that helps in the licensing discussion, but still requires to define a host compiler with options when compiling Allen's Stream library.
It's perfectly fine to have the Stream library as stand alone. I understand it needs a host compiler, but it doesn't need it to be the same as the interface code, and it doesn't need to use the same C++ standard. The only constraint is that the interface matches. Of course, if we can use C to exchange data everything is easier and stable, but so far (in 20 years of C++ development) we had only one case of ABI change, and I believe we can avoid passing std::string instances from the interface to the Stream library. In the future, after C++23, we may have other ABI changes, but I'm pretty sure they will be very easy to control.