Move getFileDescendents to NewOracleBookkeepingDB
We should start moving methods to NewOracleBookkeepingDB as we've been having some issues on the oracle side and the current queries aren't ideal from the database's perspective.
As a reminder, we have three classes: OracleBookkeepingDB, LegacyOracleBookkeepingDB and NewOracleBookkeepingDB. The idea of OracleBookkeepingDB is that it can redirect calls to one of the implementations, or potentially both if you want to compare the results from each. See here for more details.
getFileDescendents seems like a good candidate for a first method to reimplement as it looks like it can be written more efficiently (especially for groups of many LFNs). Ideally I think we should aim to replace it with a single method than runs a single query, possibly batching in to groups of LFNs if there is too many of them as the same time. We should use bind parameters rather than string manipulation to add parameters to the query. If practical can also add "unit" tests like this one.
Having taken a quick look, it might make sense to add a new method (getFileDescendentsTree?) which has a more helpful returned structure when depth>=2 then make getFileDescendents call that method and reshape the object to what the old one produced for compatibility.
@azhelezo Can you take a look? After we're happy with the strategy for migrating methods from LegacyOracleBookkeepingDB -> NewOracleBookkeepingDB method we can then decide which methods we want to move next.
Proposed interface:
# Depth = 1
{
"descendents": {
"lfn1": {"child1": {}},
"lfn2": {"child1": {}},
"lfn3": {"child2": {}},
"lfn4": {}
},
"metadata": {
"lfn1": {"got_replicas": True , "FileType": "DST"},
"lfn2": {"got_replicas": True , "FileType": "DST"}, # other metadata
"lfn3": {"got_replicas": True , "FileType": "DST"}, # other metadata
"lfn4": {"got_replicas": True , "FileType": "DST"}, # other metadata
"child1" : {"got_replicas": True , "FileType": "DST"}, # other metadata
"child2" : {"got_replicas": True , "FileType": "DST"}, # other metadata
}
}
# Depth = 2
{
"descendents":
{
"lfn1": {
"child1": {
"grandchild1": {},
},
},
"lfn2": {
"child1": {
"grandchild1": {}
}
},
"lfn3": {
"child2": {
"grandchild2": {}
}
},
"lfn4": {}
},
"metadata": {
"lfn1": {"got_replicas": True , "FileType": "DST"}, # other metadata
"lfn2": {"got_replicas": True , "FileType": "DST"}, # other metadata
"lfn3": {"got_replicas": True , "FileType": "DST"}, # other metadata
"lfn4": {"got_replicas": True , "FileType": "DST"}, # other metadata
"child1" : {"got_replicas": True , "FileType": "DST"}, # other metadata
"child2" : {"got_replicas": True , "FileType": "DST"}, # other metadata
"grandchild1" : {"got_replicas": True , "FileType": "DST"}, # other metadata
"grandchild2" : {"got_replicas": True , "FileType": "DST"}, # other metadata
}
}