Skip to content
Snippets Groups Projects

Add support to publish lists of FullPhysVol and AlignableTransform nodes by any key

The MR adds support to publish lists of FullPhysVol and AlignableTransform nodes. ( Note: This is a sister MR of https://gitlab.cern.ch/atlas/GeoModelPlugins/-/merge_requests/4, and this is requested to build the latter. )

The nodes now can be published with any type of key. At the moment, strings and integers are supported for testing, but that can be easily extended to any type.

The new tools can be used in plugins (refer to the sister MR for an example of that) as well as in plain GeoModel code (see the new GeoModelExamples/HelloToy example).

The lists of published nodes can be stored in separate DB tables, by providing custom table names (actually, suffixes: they are added to a default table name). Suffixes are built by adding the plugin's name, if any (the sister MR introduce that as well, to avoid conflicting tables' names), to the suffix provided by the client's code.

This MR adds all the needed functionalities to publish FPV and AXF nodes, and to save the lists of nodes to the DB in customized, separate tables. In a separate MR new code to read the lists of published nodes from the DB will be provided.

Edited by Riccardo Maria Bianchi

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • added 1 commit

    • fdd473fe - CI: Update Coin and SoQt sources path

    Compare with previous version

  • Riccardo Maria Bianchi changed the description

    changed the description

  • added 1 commit

    • 32b06c07 - CI: fix SoQt build after path update

    Compare with previous version

  • added 1 commit

    • 57bd5e5b - Moved GeoModelIO/GeoModelPublish functionality to GeoModelKernel/GeoPublisher,...

    Compare with previous version

  • added 2 commits

    • 3b29cfe7 - Moved GeoStore to GeoPublisher and removed inheritance from GeoVStore, it's now independent.
    • 484db385 - Fix file existance check

    Compare with previous version

  • added 3 commits

    • 646aad0a - New interface for reading back published nodes from the DB
    • 7e46bffa - Adapt to new nodes-publishing interface
    • 43a1b21b - Add new example to read back published nodes from the DB

    Compare with previous version

  • added 2 commits

    • da809a44 - empty definitions of constructor/destructor are now in header. Removing unnecessary cpp.
    • 42844815 - Move to latest changes in GeoVGeometryPlugin and GeoPublisher

    Compare with previous version

  • added 1 commit

    • 13e9d2c4 - Fix gmcat when plugin does not provide a publisher (e.g., LArPlugin)

    Compare with previous version

  • added 1 commit

    • 994f0a2d - Fix WriteGeoModel when a Publisher was provided but no nodes were published

    Compare with previous version

  • added 1 commit

    Compare with previous version

  • OK, I have implemented all the changes that came out from our meeting in August. In particular:

    • plugin constructor without required name DONE. We now have two constructors in GeoVGeometryPlugin: a default constructor, which can be used by plugins that don't need to publish lists of FullPhysVol/AlignableTransform nodes; and a parametrized constructor, accepting a plugin's name and an instance of GeoPublisher, for those plugins that need to publish lists of nodes.

    • no subclass from geovstore DONE. I now created a new class, the GeoModelKernel/GeoPublisher, which does not inherit from GeoModelKernel/GeoVStore anymore. So, GeoVStore can be used as originally intended and GeoPublisher can implement its own methods. NOTE: Because of this change, I added a third parameter to the GeoModelKernel/GeoVGeometryPlugin::create() method, to accept a pointer to an instance of GeoPublisher in addition to the pointer to a GeoVStore.

    • swap strings vs integers in examples DONE. The examples (both plugin examples and 'plain' examples) now show FullPhysVols published with integer-based keys and AlignableTransforms published with string-based keys.

    • simplify interface / remove suffixes DONE. GeoPublisher now only uses the plugin's name to create custom, dedicated DB tables to store the nodes published by the plugin itself. No more need for additional "suffixes".

    • readback methods development DONE. I designed and have implemented methods in GeoModelRead and GeoModelDBManager to get back the lists of published nodes from the DB. Those are retrieved as maps containing <key, GeoFullPhysVol*>, in case of GeoFullPhysVol, or <key, GeoAlignableTransform*>, in case of GeoAlignableTransform. **NOTE:** even if all keys are stored as things inside the DB, they are returned to clients in their original format (i.e., std::stringif strings,int` if integers, and so forth).

      NOTE: The HelloToy example shows how to publish nodes and retrieve them back from the DB: https://gitlab.cern.ch/rbianchi/GeoModel/-/tree/master-publish-list-nodes/GeoModelExamples/HelloToy

      Also, the two Toy plugins shows how to publish nodes from a plugin:

      Those two above are part of the atlas/GeoModelPlugins!4 MR.

      Briefly, clients can publish nodes as:

    for (int i=0;i<100;i++) {
        GeoFullPhysVol         *ringPhys = new GeoFullPhysVol(ringLog);
        GeoAlignableTransform  *xform    = new GeoAlignableTransform(GeoTrf::TranslateZ3D((i-50)*20*SYSTEM_OF_UNITS::cm));
        toyPhys->add(xform);
        toyPhys->add(ringPhys);
        
        // *** publish the list of FPV and AXF nodes ***
        // in this example, we use integer-based keys for FullPhysVols...
        unsigned int keyInt = i+1;
        publisher->publishFPV( ringPhys, keyInt );
        // ...and string-based keys for AlignableTransforms
        std::string keyStr = "HelloToyExample-FPV-" + std::to_string(i+1);
        publisher->publishAXF( xform, keyStr );
      }

    And then they can read them back from the DB with:

       /* setup the GeoModel reader */
      GeoModelIO::ReadGeoModel readInGeo = GeoModelIO::ReadGeoModel(db);
      std::cout << "OK! ReadGeoModel is set." << std::endl;
    
    
      /* build the GeoModel tree */
      GeoPhysVol* world = readInGeo.buildGeoModel(); // builds the whole GeoModel tree in memory
      std::cout << "ReadGeoModel::buildGeoModel() done." << std::endl;
    
      std::map<unsigned int, GeoFullPhysVol*> mapFPV = readInGeo.getPublishedNodes<unsigned int, GeoFullPhysVol*>("HelloToyExample");
      std::map<std::string, GeoAlignableTransform*> mapAXF = readInGeo.getPublishedNodes<std::string, GeoAlignableTransform*>("HelloToyExample");
    
      std::cout << "Published FullPhysVols from the DB...\n";
      unsigned int ii=0;
      for ( auto const& [key, vol] : mapFPV ) 
      {
          if(0==ii) std::cout << " [key type (compiler's code): '" << typeid(key).name() << "']\n";
          std::cout << "--> key: " << key 
                    << " - GeoFullPhysVol*: " << vol 
                    << std::endl;
          ++ii;
      }
      
      ii=0;
      std::cout << "Published AlignableTransforms from the DB...\n";
      for ( auto const& [key, xf] : mapAXF ) 
      {
          if(0==ii) std::cout << " [key type (compiler's code): '" << typeid(key).name() << "']\n";
          std::cout << "--> key: " << key 
                    << " - AlignableTransform*: " << xf 
                    << std::endl;
          ++ii;
       }
    • different lists and DB tables for different plugins Each publisher creates a custom, dedicated DB table to store its lists of published nodes. When chaining multiple plugins with gmcat, one dedicated table per plugin will be created in the output DB to store the lists of published nodes defined by any given plugin. In that way, each subsystem can have its own table containing its published nodes only. I also have modified GMCAT to accommodate to the latest changes. It now can handle multiple plugins using different publishers.

    For example, here the output of gmcat chaining two Toy example plugins:

    $ ../install/bin/gmcat ../install/lib/libToyGeometryPlugin.1.dylib ../install/lib/libToyGeometryPluginCustomPublisher.1.dylib  -o geogmcat.db
    The plugin, whose name is 'ToyGeometryPlugin', has been created.
    The plugin, whose name is 'ToyGeometryPluginCustomPublisher', has been created.
    The Geometry Database 'geogmcat.db' has been opened successfully!
    Saving the GeoModel tree to file: 'geogmcat.db'
    Info: number of Materials records to dump into the DB:5
    Info: number of Elements records to dump into the DB:11
    Info: number of NameTags records to dump into the DB:2
    Info: number of AlignableTransforms records to dump into the DB:200
    Info: number of Functions records to dump into the DB:4
    Info: number of SerialTransformers records to dump into the DB:4
    Info: number of Shapes records to dump into the DB: 9
    Info: number of SerialDenominators records to dump into the DB:6
    Info: number of PhysVols records to dump into the DB:7
    Info: number of FullPhysVols records to dump into the DB:200
    Info: number of LogVols records to dump into the DB:9
    Info: number of ChildrenPositions records to dump into the DB:416
    
    INFO: A pointer to a GeoPublisher instance has been provided, so we dump the published list of FullPhysVol and AlignableTransforms
    
    Info: number of PublishedAlignableTransforms_ToyGeometryPlugin records to dump into the DB:100
    Info: number of PublishedFullPhysVols_ToyGeometryPlugin records to dump into the DB:100
    Info: number of PublishedAlignableTransforms_ToyGeometryPluginCustomPublisher records to dump into the DB:100
    Info: number of PublishedFullPhysVols_ToyGeometryPluginCustomPublisher records to dump into the DB:100

    and the output DB will look like this, with tables dedicated to each plugin:

    $ sqlite3 geogmcat.db
    SQLite version 3.24.0 2018-06-04 14:10:15
    Enter ".help" for usage hints.
    sqlite> .tables
    AlignableTransforms
    ChildrenPositions
    Elements
    FullPhysVols
    Functions
    GeoNodesTypes
    LogVols
    Materials
    NameTags
    PhysVols
    PublishedAlignableTransforms_ToyGeometryPlugin
    PublishedAlignableTransforms_ToyGeometryPluginCustomPublisher
    PublishedFullPhysVols_ToyGeometryPlugin
    PublishedFullPhysVols_ToyGeometryPluginCustomPublisher
    RootVolume
    SerialDenominators
    SerialTransformers
    Shapes
    Transforms
    dbversion
    sqlite>

    The MR is ready to be merged, if we agree on all changes.

    As soon as this is merged, the Plugins' MR atlas/GeoModelPlugins!4 can be merged as well.

    Best, -- Ric.

    Edited by Riccardo Maria Bianchi
    • Resolved by Riccardo Maria Bianchi

      P.S. NOTE: This MR also fixes the build of Coin3D in the CI, which is present in master.

      Pinging @mbandier : following our discussion on that at today's meeting, as you can see by clicking on the CI icons above, this MR also fixes the problem with the Coin and SoQt sources. They are now stored on an ATLAS web folder (and, in fact, since we use a stable version, we don't actually need to grab something from an online repo. The same we do for all AthenaExternals.) Please let me know if you have more questions. Best, -- Ric.

      Edited by Riccardo Maria Bianchi
  • added 2 commits

    • dfb4c227 - add TODO reminder
    • 7613198b - Removed the unused GeoVStore from GeoModelKernel

    Compare with previous version

  • @tsulaia : following our discussion in today's meeting, I have just removed GeoVStore from GeoModelKernel. I also have adapted plugins to this change in atlas/GeoModelPlugins!4.

    @dellacqu : I also have adjusted the GDMLtoGM plugin to accommodate this change

    Pinging @boudreau, too, for information.

    Best, -- Ric.

    Edited by Riccardo Maria Bianchi
  • added 1 commit

    • 3a4a1b88 - Implement uniform interface for publishing nodes and reading them back. Also, updated the example.

    Compare with previous version

  • @tsulaia and @boudreau , a last note: I have just modified the API of GeoPublisher to have a uniform interface for publishing nodes and read them back.

    Now, clients can publish nodes with:

    unsigned int keyInt = i+1;
    publisher->publishNode<GeoVFullPhysVol*,unsigned>(ringPhys, keyInt);
    
    std::string keyStr = "HelloToy-AXF-" + std::to_string(i+1);
    publisher->publishNode<GeoAlignableTransform*,std::string>(xform, keyStr);

    then, they read them back from the DB with:

    std::map<unsigned int, GeoFullPhysVol*> mapFPV = readInGeo.getPublishedNodes<unsigned int, GeoFullPhysVol*>("HelloToyExample");
    
    std::map<std::string, GeoAlignableTransform*> mapAXF = readInGeo.getPublishedNodes<std::string, GeoAlignableTransform*>("HelloToyExample");

    In this way, it is more clear/explicit what key type is used when publishing and what key type is retrieving when getting them back.

    I also have updated the examples and the plugins accordingly.

    Let me know if you have any comments/questions, of course.

    (Later, I will implement an internal check to print a warning if the type defined at retrieval is different from the type stored while publishing.)

    Edited by Riccardo Maria Bianchi
  • Hi @rbianchi , this MR has picked some conflicts

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading