Skip to content
Snippets Groups Projects
Commit bb2c747a authored by Frank Winklmeier's avatar Frank Winklmeier
Browse files

Merge branch 'atn_summary_table_3' into '21.0' 

Fixes to ATN summary table for git releases (ATR-16108)

See merge request !864

Former-commit-id: d208f359
parents 5d0bf6d7 7fbf2c69
No related branches found
No related tags found
112 merge requests!76035Updated rel21 number,!66713update MadGraph version,!63304adding missing electron iso WPs (Tight_VarRad, Loose_VarRad, TightTrackOnly_VarRad),!62053Merge branch 'sh_xrd_warning' into 'master',!61521fix maxPVrefit,!59663fixed a typo in HIGG8D1 format (followup of https://gitlab.cern.ch/atlas/athena/-/merge_requests/59575),!51680Add functionality to MultiBjetFilter,!50754Return TruthTaus from truth matching tool in SUSY tools,!48850Added new HDBS derivation (HDBS3),!46643Draft:Adding MET Significance in AT,!46457Draft: Adding the tools to run over data,!46454Draft: Adding the tools to be able to run over data,!46320Draft: Adding support for Leading3SignalJets strategy,!45689Bug fix: Use wrong vector to calculate largest bin number in findBin(),!45211Update in the FFJetSmearingTool,!45045WIP: Include another condition for hit merging in FCS_StepInfoSD.cxx,!45043Merge branch 'Update-DAOD_IDNCB' into '21.2',!44869Draft: Update to candidate store,!44864Draft: WIP- MuonUncorrelateSystematics,!44215added ART tests,!43718Modification of EXOT17.py for fractionally charged particle search,!42752Fix for TopRecon UserHook,!4217921.9-first_steps-InDetTrackingGeometryXML,!42171Add LAr cell DQ plots to UPC stream,!41097TruthJetFilter phi filter,!40581Fix crash in egamma topocluster decorator,!40342WIP: ATR-22546 - add missing trigger to MC16a,!39592Cherry picking !27086 in 21.0 mc16d (ATLPHYSVAL-705),!39162Draft: Insert BCM' support,!38765Bis78 cabling,!38102Add SameParticleHardScatteringFilter,!37848Add TrigRingerRings to EGAM3, EGAM4 and EGAM9,!37483Fix in monitoring of Pixel FE errors,!36893Fix size of RPC active region in BIS78 and strip material,!36766Truth class fallback max dr,!36648Rivet doc update,!3645821.9: Improving material map description (ATLITKSW-127),!36293WIP pixel updates,!35816Add HardScatterParticles into the EXOT8 Derivation,!35273Fix JETM1: add back TruthEvents,!34993KF-input adding jXERHO to the AOD outputs,!34864LH rings in front ot HR rigs,!34763Fix L1_4jJ15.0ETA25 item definition (ATR-21261),!3473421.0 fix dq servers,!34244WIP: 21.2 topq1 add rc jets 2020 06 27,!34182Merge new MCTruthClassifier developments into master,!34164Use allMuons instead oc combinedOnly for BPHY15,!33996WIP: Correct MM zpositions,!33707WIP: Fixes for the updated Powheg Wt_DR process, and removal of Wt_DR_onshellW and Wt_DS_onshellW processes,!33109Add delta-R to nearest VR track jet decoration,!33012Adding Prompt Lepton Improved Veto to TOPQ,!3297121.6 parameter dict pythia fixes,!3291021.3 salva s0,!3288421.0 salva s8,!32637WIP: Add tracks in R=0.4 jets to STDM11,!32353WIP: Fix rare bug in jet smearing RNG seed,!3217421.9 - Bug fix for test stream creation in RD53BEncodingTool,!32173H7 lhe interface,!32051Fix of compilation warnings in CPAnalysisExamples,!31371Add missing copyright headings,!31361WIP: AnalysisTop Update - New Muon Isolation WPs included,!3135321.3,!3134821.9 README Update,!31345Add HEC sampling fractions macros,!31136added also PUsub for LargeR jets,!31123WIP: Master ttmdev,!30796Reducing number of warnings in MuonEfficiencyCorrections when missing dRJet decoration,!3043421.6 mg,!3036121.9 fixing bug with layer indices in PixelRDOAnalysis,!30295Move from boost::filesystem to std::filesystem,!29617IDPerfmonZmumu updates,!29454Idpvm 21.0 fix typo in trackselection config,!29180WIP: Add multiplicity calculations to JetEtMiss code,!29085TRT modification for fractional charged particles,!28623Overlay as part of FastChain transform in 21.3,!28497ART tests - fixing the bugs,!27685update MG version to 2.6.7p3.atlas4,!27668Massimos 21.3 bis78,!27299WIP: Turning b-jet thinning off in SUSY1 (permanent?),!27254WIP: Exclusive jets request -- Fix ExclusiveJets algo / ATR-17320,!27094JETM8 UFO fix,!26790Fixes for Simulation ART jobs (ATLASSIM-4309),!26747Update frozen shower generation scripts and code to be compatible with recent developments,!26641Rename member functions to start with lower case,!26592Master upload 2018 global monitoring scripts p2,!2618221.0 z counting,!25405Add new ART test: test_physics_pp_v8_chainorder_build.sh,!2498621.6 transf,!24487Time-stamped PFlow containers in output DAOD,!24347Electron-in-jet and overlap removal interplay fixed,!24210WIP: Add syst for r21 vs r20.7 differences to defaults,!2420321.3 l1 muon 00 - new MuCTPI_RDO.h,!24037Make DCS Calculator dependent on sqlalchemy; fix python code quality issues,!23744New trigger derivation for jet substructure,!23377WIP pixel ID scheme for ITk,!23330Inner Detector create misalignments: radial distortions,!2325521.0 z counting,!22695Master salva id perf mon,!2230421.0-ART_test-InDetPhysValMonitoring,!22214add an utility function to TrigT1TGCRecRoiSvc for storing ROI position and the eta/phi size,!22172changes related to 1-3 points in ATR-19579,!22124Changes related to ticket 19579,!22118Remove cabling information for new thresholds,!21684WIP: test push into tdaq-htt,!21621Updating fJVT WP with new recommendations in TopCPTools,!2153821.0-ART_test-InDetPhysValMonitoring,!21325Tilecal new laser unpack 2019 02 19,!21304WIP: Fix ATR-19377 L1Topo vbs alg,!2117721.3 po bugfixes,!21039Merging in features for HTT Bank Generation,!2089821.2 rpv1l,!20518Mydev
Hide whitespace changes
Inline Side-by-side
Trigger/TrigValidation/TrigValTools/bin/atn_summary.pl
+ 90
− 135
View file @ bb2c747a
#!/usr/bin/perl -w
use strict;
use File::Basename;
use Date::Manip::Date;
# Produce summary table for trigtest.pl tests in same dir or first parameter to script
my $output="index.html";
......@@ -85,7 +86,7 @@ sub main(){
system("printenv > atn_summary_printenv.log");
print "\nTo test, set the following environment. If unset, results may not reproduce those within ATN. Read this script for more details on testing procedure.\n";
my $ev;
foreach $ev qw(AtlasProject NICOS_NIGHTLY_NAME NICOS_ATLAS_PROJECT NICOS_ATLAS_RELEASE NICOS_PROJECT_RELNAME_COPY){
foreach $ev qw(AtlasProject AtlasVersion NICOS_GIT_RELBRANCH NICOS_PROJECT_RELNAME_COPY){
print "export " . $ev . "=" . getEnv($ev,"") . "\n";
}
print "\n";
......@@ -97,60 +98,63 @@ sub main(){
my %doc=();
my %timer=();
my %order = getorder();
my $project = getEnv('AtlasProject',getEnv('NICOS_ATLAS_PROJECT',''));
my $gitbranch = getEnv('NICOS_GIT_RELBRANCH','');
my $release = getEnv('AtlasVersion','');
opendir(DIRHANDLE, $testdir) || die "Cannot opendir $testdir: $!";
foreach my $name (sort readdir(DIRHANDLE)) {
if (-d $name and $name !~ /^\./){
print "found dir: $name ... ";
push @testnames, $name;
if (-f "$name/$summaryout"){
print "found summary file\n";
open SUMMARYIN, "<$name/$summaryout";
while (<SUMMARYIN>){
if (/exitcode = (\d+)/){
$exitcodes{$name} = $1;
}
elsif (/maskedcode = (\d+)/){
$maskedcodes{$name} = $1;
}
}
close SUMMARYIN;
if (-f "$name/$timerout"){
print "found timer file\n";
open TIMERIN, "<$name/$timerout";
while (<TIMERIN>){
my ($wall,$user,$system)=split;
$timer{$name}=$wall;
}
close TIMERIN;
}
else {
print "no timer file\n";
}
} else {
print "no summary file\n";
}
$doc{$name}="";
if (-f "$name/$docout"){
print "found doc file\n";
open DOCIN, "<$name/$docout";
while (<DOCIN>){
chomp;
$doc{$name}.=$_;
}
close DOCIN;
}
else {
print "no doc file\n";
}
}
if (-d $name and $name !~ /^\./){
print "found dir: $name ... ";
push @testnames, $name;
if (-f "$name/$summaryout"){
print "found summary file\n";
open SUMMARYIN, "<$name/$summaryout";
while (<SUMMARYIN>){
if (/exitcode = (\d+)/){
$exitcodes{$name} = $1;
}
elsif (/maskedcode = (\d+)/){
$maskedcodes{$name} = $1;
}
}
close SUMMARYIN;
if (-f "$name/$timerout"){
print "found timer file\n";
open TIMERIN, "<$name/$timerout";
while (<TIMERIN>){
my ($wall,$user,$system)=split;
$timer{$name}=$wall;
}
close TIMERIN;
}
else {
print "no timer file\n";
}
} else {
print "no summary file\n";
}
$doc{$name}="";
if (-f "$name/$docout"){
print "found doc file\n";
open DOCIN, "<$name/$docout";
while (<DOCIN>){
chomp;
$doc{$name}.=$_;
}
close DOCIN;
}
else {
print "no doc file\n";
}
}
}
closedir(DIRHANDLE);
open HTMLOUT, "> $output" or die "failed to write to $output: $!";
print HTMLOUT "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">
<html>\n<head>\n<script type=\"text/javascript\" src=\"http://atlas-project-trigger-release-validation.web.cern.ch/atlas-project-trigger-release-validation/www/js/sorttable.js\"></script>\n<script type=\"text/javascript\" src=\"http://atlas-project-trigger-release-validation.web.cern.ch/atlas-project-trigger-release-validation/www/js/suitehighlight.min.js\"></script>\n<meta http-equiv=\"Content-Type\" content=\"text/html;charset=utf-8\" >
<title>Trigger ATN results for ".getEnv("AtlasProject","").",".getEnv("NICOS_NIGHTLY_NAME","").",$atn_rel</title>\n
<title>Trigger ATN results for $project,$gitbranch,$release</title>\n
<script type=\"text/javascript\">
var thisRelease = null;
......@@ -169,7 +173,7 @@ function highlightDiffs(clear) {
var diffs = 0;
for (var i=1; i < table1.rows.length; i++) {
var cells = table1.rows[i].cells;
for (var j=2; j < cells.length; j++) { // skip name and wiki column
for (var j=1; j < cells.length; j++) { // skip name
if (j==9) continue; // skip timing
if (!table2 || cells[j].textContent == table2.rows[i].cells[j].textContent) {
cells[j].style.backgroundColor = '';
......@@ -245,103 +249,63 @@ function showBuildFailures(failures,link) {
";
my $date=qx(date);
my $now=qx(date);
my $testWWWPage=getTestWWWPage();
my $nicosWWWPage=dirname($testWWWPage);
#print ">>> nicosWWWPage=$nicosWWWPage\ntestWWWPage=$testWWWPage\n";
print HTMLOUT "<p>Last updated $date</p>\n";
if ($context){
# supported projects: Offline + AtlasHLT for bugfix, P1HLT + Tier0 for caches. Better to use case logic but this will do for now.
my $project="AtlasOffline";
if (getEnv('NICOS_ATLAS_PROJECT','') =~ /AtlasHLT|AtlasP1HLT|AtlasCAFHLT|AtlasTier0|AtlasProduction|TrigMC|AtlasTestHLT/){
$project=getEnv('NICOS_ATLAS_PROJECT','');
}
print "NICOS_ATLAS_PROJECT=getEnv('NICOS_ATLAS_PROJECT','') project=$project\n";
print HTMLOUT "<p>Last updated $now</p>\n";
my $atn_relno = $atn_rel;
$atn_relno =~ s/rel_//;
print HTMLOUT "<p><a href=\"".$testWWWPage."\">Nightly test</a>: ".getEnv("AtlasProject","").",".getEnv("NICOS_NIGHTLY_NAME","").",<b>$atn_rel</b> ($atn_platform)";
print HTMLOUT "<p><a href=\"".$testWWWPage."\">Nightly test</a>: $project,$gitbranch,<b>$release</b> ($atn_platform)";
print HTMLOUT "<script src=\"".$nicosWWWPage."/build_failures_".$atn_relno.".js\" language=\"JavaScript\"></script>\n";
print HTMLOUT "<script type=\"text/javascript\">showBuildFailures(failures_".$atn_relno."(),"."\"".$nicosWWWPage."/nicos_buildsummary_".$atn_relno.".html\")</script>";
# Link to GitLab diff between today's and yesterday's release
my $fmt="%Y-%m-%dT%H%M";
my $date = new Date::Manip::Date;
$date->parse_format($fmt,$release);
my $delta = $date->new_delta();
$delta->set(d => -1); # minus 1 day
my $yesterday = $date->calc($delta);
my $prevrel = $yesterday->printf($fmt);
my $gitdiff = "https://gitlab.cern.ch/atlas/athena/compare/nightly%2F$gitbranch%2F$prevrel...nightly%2F$gitbranch%2F$release";
print HTMLOUT "<br>Other nightlies: ";
if (defined $no_of_nightlies_exceptions{$project}){
$no_of_nightlies=$no_of_nightlies_exceptions{$project};
print "$project exceptionally has $no_of_nightlies nightly builds\n";
} else {
print "$project normally has $no_of_nightlies nightly builds\n";
}
if (defined $no_of_nightlies_exceptions{$project}){
$no_of_nightlies=$no_of_nightlies_exceptions{$project};
print "$project exceptionally has $no_of_nightlies nightly builds\n";
} else {
print "$project normally has $no_of_nightlies nightly builds\n";
}
my @dow = ('Sun','Mon','Tue','Wed','Thu','Fri','Sat');
for (my $i=0; $i<$no_of_nightlies; $i++){
if ("rel_$i" eq $atn_rel){
print HTMLOUT " $i ";
print HTMLOUT " $dow[$i] ";
} else {
my $p=getrelpath();
$p =~ s/rel_\d/rel_$i/;
$p =~ s/rel_nightly/rel_$i/;
print HTMLOUT " <a href=\"$p\">$i</a> ";
my $p=getrelpath();
$p =~ s/rel_\d/rel_$i/;
$p =~ s/rel_nightly/rel_$i/;
print HTMLOUT " <a href=\"$p\">$dow[$i]</a> ";
}
}
# useful link to nightly change notes page
my $release=getEnv('NICOS_ATLAS_RELEASE','');
my $val="";
print "NICOS_NIGHTLY_NAME=getEnv('NICOS_NIGHTLY_NAME','')\n";
if (getEnv('NICOS_NIGHTLY_NAME','') =~ /-VAL/){
$val="-val";
}
#my $cnurl="http://cern.ch/atlas-project-trigger-release-validation/www/ReleaseNotes/test/TestReleaseDiff-$release-$atn_rel$val.html";
my $cnurl="http://atlas-project-trigger-release-validation.web.cern.ch/atlas-project-trigger-release-validation/www/ReleaseNotes/ChangeNotes$project-$release-$atn_rel$val.html";
print "cnurl=$cnurl\n";
print HTMLOUT "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href=\"$cnurl\">nightly tag diffs</a>
<a href=\"$gitdiff\">GitLab diff</a>
&nbsp;&nbsp;&nbsp;&nbsp;Diff results: <select name=\"rel\" size=\"1\" onchange=\"loadPage()\">
<option value='rel_0'>rel_0</option>
<option value='rel_1'>rel_1</option>
<option value='rel_2'>rel_2</option>
<option value='rel_3'>rel_3</option>
<option value='rel_4'>rel_4</option>
<option value='rel_5'>rel_5</option>
<option value='rel_6'>rel_6</option>
<option value='rel_0'>Sun</option>
<option value='rel_1'>Mon</option>
<option value='rel_2'>Tue</option>
<option value='rel_3'>Wed</option>
<option value='rel_4'>Thu</option>
<option value='rel_5'>Fri</option>
<option value='rel_6'>Sat</option>
</select> <span id='nDiffs' style='font-weight:bold'></span>
</form><script type=\"text/javascript\">setRelease();</script></p>";
}
#my $cmt = $ENV{'CMTPATH'};
#my $nightlykey = 'builds/nightlies/';
#my $keylength = length $nightlykey;
#my $relstart = rindex($cmt,$nightlykey) ;
#if( $relstart > -1 ) {
# $relstart = $relstart + $keylength;
# # $nightly should be dev,devval, bugfix, val, ect.
# $nightly = substr $cmt, $relstart;
# my $nextslash = index $nightly, '/' ;
# $nightly = substr $nightly, 0, $nextslash ;
#}
my $nightlywiki = getEnv('NICOS_NIGHTLY_NAME','');
print "Nightly wiki page name postfix before deletes: $nightlywiki\n";
#remove val from the name
$nightlywiki =~ s/val//;
$nightlywiki =~ s/VAL//;
# remove any . from the nightly name (Wiki doesn't like these)
$nightlywiki =~ s/\.//g;
# remove any - from the nightly name (Wiki dosen't like these either)
$nightlywiki =~ s/-//g;
# Strip all but first two characters so that the version
# becomes just '16' rather than 16.X.Y etc. Only do this
# for versions starting with 16 (or higher) so that older
# versions remain with the old links
my $shortver = substr $nightlywiki, 0, 2;
if ($shortver ge 16) {
$nightlywiki=$shortver;
}
print "Nightly wiki page name using postfix: $nightlywiki\n";
print HTMLOUT "\n<table class=\"sortable\" id=\"ATNResults\" border=1><thead><tr>
<th title=\"Click to sort by test suite\">Test name</th>
<th>Wiki</th>
<th>Overall Result</th>
<th title=\"exit code of Athena\">Athena exit</th>
<th title=\"presence of any error messages in log\">Error Msgs</th>
......@@ -496,11 +460,6 @@ function showBuildFailures(failures,link) {
print "logfile=$logfile\n";
my $thename = $name;
my $shortname = $name;
# remove any . from the nightly name (Wiki doesn't like these)
$shortname =~ s/\.//g;
#truncate to desired 32 character wiki limit
$shortname = substr($shortname,0,32);
my $logurl;
if ($context){
$logurl = getlogpath($name,$testdir);
......@@ -519,7 +478,7 @@ function showBuildFailures(failures,link) {
if (exists $order{$name}) {
$order = "suite=\"$order{$name}[1]\" sorttable_customkey=\"$order{$name}[0]\"";
}
print HTMLOUT "<tr class=\"hi\"><td $order title=\"$doc{$name}\">$thename</td> <td><A HREF=\"https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerValidationATNStatus$nightlywiki#$shortname\" onClick=\"return popup(this, 'notes')\">wiki</A></td> <td>$atn_script</td> <td>$athena_exit</td> <td>$error</td> <td>$warn</td> <td style=\"white-space:nowrap\">$reg_tests</td> <td style=\"white-space:nowrap\">$rootcomp</td> <td>$checkcount</td> <td><a type=\"text/plain\" href=\"$name/$timerout\">$timer</a></td> <td>$exitcode/$maskedcode</td> <td>$postrc</td> <td><a href=\"$name\">dir</a></td><td>$logfile</td></tr>\n";
print HTMLOUT "<tr class=\"hi\"><td $order title=\"$doc{$name}\">$thename</td> <td>$atn_script</td> <td>$athena_exit</td> <td>$error</td> <td>$warn</td> <td style=\"white-space:nowrap\">$reg_tests</td> <td style=\"white-space:nowrap\">$rootcomp</td> <td>$checkcount</td> <td><a type=\"text/plain\" href=\"$name/$timerout\">$timer</a></td> <td>$exitcode/$maskedcode</td> <td>$postrc</td> <td><a href=\"$name\">dir</a></td><td>$logfile</td></tr>\n";
}
print HTMLOUT "</tbody></table>";
if (-e "$atn_timeline"){
......@@ -589,11 +548,7 @@ sub getcontext(){
print "$cwd\n";
if ($cwd =~ /NICOS_.*test([\w\d]+)\//){ # NICOS_qmtest (old) or NICOS_atntest (new) directory
$atn_platform=$1;
#print "$atn_platform\n";
#$atn_log_dir="../../NICOS_TestLog$atn_platform";
#print "atn log dir: $atn_log_dir\n";
#system("ls $atn_log_dir");
print "ATN_JOB_LOGFILE=" . getEnv("ATN_JOB_LOGFILE","") . "\n";
print "ATN_JOB_LOGFILE=" . getEnv("ATN_JOB_LOGFILE","") . "\n";
print "ATN_PACKAGE=" . getEnv("ATN_PACKAGE","") . "\n";
print "--------------------------------------------------------\n";
}
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment