present unpacked data frame as multi indexed
Summary
Presenting column names as tuples in form like
('FIELD NAME', 'BLOCK TYPE', 'BLOCK LEVEL', slot, link, position)
where:
-
'FIELD NAME'- name of the data field in format -
'BLOCK TYPE'- name of data block (one of['CDF', 'AMC13', 'AMC', 'GEB', 'VFAT']) * optional -
'BLOCK LEVEL'- one of['HEADER', 'PAYLOAD', 'TRAILER']* optional - slot, link, position - hardware topology tags with default values equal to
Nonewhere they are not applicable
would allow automated conversion of the resulting data frame columns to pandas.MultiIndex. Subsequent data analysis would benefit of easy data slicing/selection.
What is the expected correct behavior?
Performance impact expected to be minor, the data will become multi-indexed and easily accessible for analysis.
Relevant logs and/or screenshots
Simple example:
l=[('Latency', -1,-1,-1), ('VFAT HIT', 1, 0, 0), ('VFAT HIT',1, 0, 2), ('VFAT HIT',1,0,3), ('VFAT HIT', 1, 1, 0), ('VFAT HIT', 1, 1, 2)]
df = pd.DataFrame(np.random.randint(0,128,(5,6)), columns = l)
df
| (Latency, -1, -1, -1) | (VFAT HIT, 1, 0, 0) | (VFAT HIT, 1, 0, 2) | (VFAT HIT, 1, 0, 3) | (VFAT HIT, 1, 1, 0) | (VFAT HIT, 1, 1, 2) | |
|---|---|---|---|---|---|---|
| 0 | 8 | 7 | 13 | 69 | 18 | 97 |
| 1 | 6 | 94 | 54 | 77 | 75 | 52 |
| 2 | 25 | 24 | 123 | 78 | 56 | 14 |
| 3 | 92 | 103 | 68 | 60 | 6 | 98 |
| 4 | 46 | 1 | 37 | 12 | 54 | 124 |
df.columns = pd.MultiIndex.from_tuples(df.columns, names=['reg_name','slot','link','position'])
df
| reg_name | Latency | VFAT HIT | ||||
|---|---|---|---|---|---|---|
| slot | -1 | 1 | ||||
| link | -1 | 0 | 1 | |||
| position | -1 | 0 | 2 | 3 | 0 | 2 |
| 0 | 8 | 7 | 13 | 69 | 18 | 97 |
| 1 | 6 | 94 | 54 | 77 | 75 | 52 |
| 2 | 25 | 24 | 123 | 78 | 56 | 14 |
| 3 | 92 | 103 | 68 | 60 | 6 | 98 |
| 4 | 46 | 1 | 37 | 12 | 54 | 124 |
df.iloc[:, df.columns.get_level_values(2)>=0]
| reg_name | VFAT HIT | ||||
|---|---|---|---|---|---|
| slot | 1 | ||||
| link | 0 | 1 | |||
| position | 0 | 2 | 3 | 0 | 2 |
| 0 | 7 | 13 | 69 | 18 | 97 |
| 1 | 94 | 54 | 77 | 75 | 52 |
| 2 | 24 | 123 | 78 | 56 | 14 |
| 3 | 103 | 68 | 60 | 6 | 98 |
| 4 | 1 | 37 | 12 | 54 | 124 |
df.xs(0, level='link', axis = 1)
| reg_name | VFAT HIT | ||
|---|---|---|---|
| slot | 1 | ||
| position | 0 | 2 | 3 |
| 0 | 7 | 13 | 69 |
| 1 | 94 | 54 | 77 |
| 2 | 24 | 123 | 78 |
| 3 | 103 | 68 | 60 |
| 4 | 1 | 37 | 12 |