Multithreading: Check Memory Usage in Detail
The following discussion from !231 (merged) should be addressed:
-
@simonspa started a discussion: (+6 comments) Differen topic, new discussion: I'm experiencing some rather large memory leaks using a setup which reads in data using
ROTOObjectReader
. However, Valgrind's output on the leaks is not quite conclusive to me because it's obstructed by ROOT/Cling logic.@kwolters maybe you can make more sense from this?
I'm also not sure if it's really the
ROOTObjectReader
==16302== 393,228 bytes in 1 blocks are possibly lost in loss record 13,775 of 13,790 ==16302== at 0x483DD99: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==16302== by 0x157EBEA2: llvm::StringMapImpl::RehashTable(unsigned int) (in /opt/root/lib/libCling.so) ==16302== by 0x13A3BF0B: clang::ASTReader::DecodeIdentifierInfo(unsigned int) (in /opt/root/lib/libCling.so) ==16302== by 0x13A4A675: clang::ASTReader::ReadDeclarationName(clang::serialization::ModuleFile&, llvm::SmallVector<unsigned long, 64u> const&, unsigned int&) (in /opt/root/lib/libCling.so) ==16302== by 0x13A6F994: clang::ASTDeclReader::VisitDeclaratorDecl(clang::DeclaratorDecl*) (in /opt/root/lib/libCling.so) ==16302== by 0x13A7D4C5: clang::ASTDeclReader::VisitFunctionDecl(clang::FunctionDecl*) (in /opt/root/lib/libCling.so) ==16302== by 0x13A7E858: clang::ASTDeclReader::VisitCXXMethodDecl(clang::CXXMethodDecl*) (in /opt/root/lib/libCling.so) ==16302== by 0x13A87672: clang::ASTDeclReader::Visit(clang::Decl*) (in /opt/root/lib/libCling.so) ==16302== by 0x13A87C30: clang::ASTReader::ReadDeclRecord(unsigned int) (in /opt/root/lib/libCling.so) ==16302== by 0x13A0CC45: clang::ASTReader::GetDecl(unsigned int) [clone .part.0] (in /opt/root/lib/libCling.so) ==16302== by 0x13A3B087: clang::ASTReader::FindExternalLexicalDecls(clang::DeclContext const*, llvm::function_ref<bool (clang::Decl::Kind)>, llvm::SmallVectorImpl<clang::Decl*>&) (in /opt/root/lib/libCling.so) ==16302== by 0x13B45349: clang::MultiplexExternalSemaSource::FindExternalLexicalDecls(clang::DeclContext const*, llvm::function_ref<bool (clang::Decl::Kind)>, llvm::SmallVectorImpl<clang::Decl*>&) (in /opt/root/lib/libCling.so) ==16302== ==16302== 474,598 bytes in 56 blocks are possibly lost in loss record 13,777 of 13,790 ==16302== at 0x483C0F3: operator new(unsigned long, std::nothrow_t const&) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==16302== by 0x157D5AD7: llvm::MemoryBuffer::getNewUninitMemBuffer(unsigned long, llvm::Twine const&) (in /opt/root/lib/libCling.so) ==16302== by 0x157D5C33: llvm::MemoryBuffer::getMemBufferCopy(llvm::StringRef, llvm::Twine const&) (in /opt/root/lib/libCling.so) ==16302== by 0x13A10A9B: clang::ASTReader::ReadSLocEntry(int)::{lambda(llvm::BitstreamCursor&, llvm::StringRef)#1}::operator()(llvm::BitstreamCursor&, llvm::StringRef) const [clone .isra.0] (in /opt/root/lib/libCling.so) ==16302== by 0x13A316AB: clang::ASTReader::ReadSLocEntry(int) [clone .part.0] (in /opt/root/lib/libCling.so) ==16302== by 0x13A31977: non-virtual thunk to clang::ASTReader::ReadSLocEntry(int) (in /opt/root/lib/libCling.so) ==16302== by 0x1556C519: clang::SourceManager::loadSLocEntry(unsigned int, bool*) const (in /opt/root/lib/libCling.so) ==16302== by 0x1556CAE4: clang::SourceManager::getFileIDLoaded(unsigned int) const [clone .part.0] (in /opt/root/lib/libCling.so) ==16302== by 0x1557242D: clang::SourceManager::getCharacterData(clang::SourceLocation, bool*) const (in /opt/root/lib/libCling.so) ==16302== by 0x154AC041: clang::Lexer::getSpelling[abi:cxx11](clang::Token const&, clang::SourceManager const&, clang::LangOptions const&, bool*) (in /opt/root/lib/libCling.so) ==16302== by 0x154BF041: clang::MacroInfo::isIdenticalTo(clang::MacroInfo const&, clang::Preprocessor&, bool) const (in /opt/root/lib/libCling.so) ==16302== by 0x15508FBF: clang::Preprocessor::updateModuleMacroInfo(clang::IdentifierInfo const*, clang::Preprocessor::ModuleMacroInfo&) (in /opt/root/lib/libCling.so) ==16302== ==16302== 584,981 bytes in 161 blocks are possibly lost in loss record 13,780 of 13,790 ==16302== at 0x483C0F3: operator new(unsigned long, std::nothrow_t const&) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==16302== by 0x157D5AD7: llvm::MemoryBuffer::getNewUninitMemBuffer(unsigned long, llvm::Twine const&) (in /opt/root/lib/libCling.so) ==16302== by 0x157D5C33: llvm::MemoryBuffer::getMemBufferCopy(llvm::StringRef, llvm::Twine const&) (in /opt/root/lib/libCling.so) ==16302== by 0x13A10A9B: clang::ASTReader::ReadSLocEntry(int)::{lambda(llvm::BitstreamCursor&, llvm::StringRef)#1}::operator()(llvm::BitstreamCursor&, llvm::StringRef) const [clone .isra.0] (in /opt/root/lib/libCling.so) ==16302== by 0x13A316AB: clang::ASTReader::ReadSLocEntry(int) [clone .part.0] (in /opt/root/lib/libCling.so) ==16302== by 0x13A31977: non-virtual thunk to clang::ASTReader::ReadSLocEntry(int) (in /opt/root/lib/libCling.so) ==16302== by 0x1556C519: clang::SourceManager::loadSLocEntry(unsigned int, bool*) const (in /opt/root/lib/libCling.so) ==16302== by 0x1556C7A8: clang::SourceManager::isOffsetInFileID(clang::FileID, unsigned int) const (in /opt/root/lib/libCling.so) ==16302== by 0x1556CB0D: clang::SourceManager::getFileIDLoaded(unsigned int) const [clone .part.0] (in /opt/root/lib/libCling.so) ==16302== by 0x1556D640: clang::SourceManager::getFileIDSlow(unsigned int) const (in /opt/root/lib/libCling.so) ==16302== by 0x13A3444A: clang::ASTReader::ReadPragmaDiagnosticMappings(clang::DiagnosticsEngine&) (in /opt/root/lib/libCling.so) ==16302== by 0x13A4A7C3: clang::ASTReader::InitializeContext() (in /opt/root/lib/libCling.so) ==16302== ==16302== 965,891 bytes in 74 blocks are possibly lost in loss record 13,784 of 13,790 ==16302== at 0x483C0F3: operator new(unsigned long, std::nothrow_t const&) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==16302== by 0x157D5AD7: llvm::MemoryBuffer::getNewUninitMemBuffer(unsigned long, llvm::Twine const&) (in /opt/root/lib/libCling.so) ==16302== by 0x157D5C33: llvm::MemoryBuffer::getMemBufferCopy(llvm::StringRef, llvm::Twine const&) (in /opt/root/lib/libCling.so) ==16302== by 0x13A10A9B: clang::ASTReader::ReadSLocEntry(int)::{lambda(llvm::BitstreamCursor&, llvm::StringRef)#1}::operator()(llvm::BitstreamCursor&, llvm::StringRef) const [clone .isra.0] (in /opt/root/lib/libCling.so) ==16302== by 0x13A313C1: clang::ASTReader::ReadSLocEntry(int) [clone .part.0] (in /opt/root/lib/libCling.so) ==16302== by 0x13A31977: non-virtual thunk to clang::ASTReader::ReadSLocEntry(int) (in /opt/root/lib/libCling.so) ==16302== by 0x1556C519: clang::SourceManager::loadSLocEntry(unsigned int, bool*) const (in /opt/root/lib/libCling.so) ==16302== by 0x13A535B7: clang::ASTReader::ReadAST(llvm::StringRef, clang::serialization::ModuleKind, clang::SourceLocation, unsigned int, llvm::SmallVectorImpl<clang::ASTReader::ImportedSubmodule>*) (in /opt/root/lib/libCling.so) ==16302== by 0x1378CD18: clang::CompilerInstance::loadModule(clang::SourceLocation, llvm::ArrayRef<std::pair<clang::IdentifierInfo*, clang::SourceLocation> >, clang::Module::NameVisibilityKind, bool) (in /opt/root/lib/libCling.so) ==16302== by 0x1551F760: (anonymous namespace)::PragmaModuleImportHandler::HandlePragma(clang::Preprocessor&, clang::PragmaIntroducerKind, clang::Token&) (in /opt/root/lib/libCling.so) ==16302== by 0x15519DE3: clang::Preprocessor::HandlePragmaDirective(clang::SourceLocation, clang::PragmaIntroducerKind) (in /opt/root/lib/libCling.so) ==16302== by 0x154F2B68: clang::Preprocessor::HandleDirective(clang::Token&) (in /opt/root/lib/libCling.so) ==16302== ==16302== 1,123,454 bytes in 447 blocks are possibly lost in loss record 13,786 of 13,790 ==16302== at 0x483C0F3: operator new(unsigned long, std::nothrow_t const&) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==16302== by 0x157D5AD7: llvm::MemoryBuffer::getNewUninitMemBuffer(unsigned long, llvm::Twine const&) (in /opt/root/lib/libCling.so) ==16302== by 0x157D5C33: llvm::MemoryBuffer::getMemBufferCopy(llvm::StringRef, llvm::Twine const&) (in /opt/root/lib/libCling.so) ==16302== by 0x13A10A9B: clang::ASTReader::ReadSLocEntry(int)::{lambda(llvm::BitstreamCursor&, llvm::StringRef)#1}::operator()(llvm::BitstreamCursor&, llvm::StringRef) const [clone .isra.0] (in /opt/root/lib/libCling.so) ==16302== by 0x13A316AB: clang::ASTReader::ReadSLocEntry(int) [clone .part.0] (in /opt/root/lib/libCling.so) ==16302== by 0x13A31977: non-virtual thunk to clang::ASTReader::ReadSLocEntry(int) (in /opt/root/lib/libCling.so) ==16302== by 0x1556C519: clang::SourceManager::loadSLocEntry(unsigned int, bool*) const (in /opt/root/lib/libCling.so) ==16302== by 0x1556CAE4: clang::SourceManager::getFileIDLoaded(unsigned int) const [clone .part.0] (in /opt/root/lib/libCling.so) ==16302== by 0x1556D640: clang::SourceManager::getFileIDSlow(unsigned int) const (in /opt/root/lib/libCling.so) ==16302== by 0x13A3444A: clang::ASTReader::ReadPragmaDiagnosticMappings(clang::DiagnosticsEngine&) (in /opt/root/lib/libCling.so) ==16302== by 0x13A4A7C3: clang::ASTReader::InitializeContext() (in /opt/root/lib/libCling.so) ==16302== by 0x13A5412E: clang::ASTReader::ReadAST(llvm::StringRef, clang::serialization::ModuleKind, clang::SourceLocation, unsigned int, llvm::SmallVectorImpl<clang::ASTReader::ImportedSubmodule>*) (in /opt/root/lib/libCling.so) ==16302== ==16302== 1,184,344 (40,600 direct, 1,143,744 indirect) bytes in 203 blocks are definitely lost in loss record 13,788 of 13,790 ==16302== at 0x483BE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==16302== by 0xE64FCED: TStorage::ObjectAlloc(unsigned long) (in /opt/root/lib/libCore.so) ==16302== by 0xD53A235: operator new (TObject.h:152) ==16302== by 0xD53A235: ROOT::new_allpixcLcLPixelCharge(void*) (AllpixObjectsDictionary.cxx:2392) ==16302== by 0xE6ED6D6: TClass::New(TClass::ENewType, bool) const (in /opt/root/lib/libCore.so) ==16302== by 0xDD3710B: TBufferFile::ReadObjectAny(TClass const*) (in /opt/root/lib/libRIO.so) ==16302== by 0xDD807B8: TEmulatedCollectionProxy::ReadItems(int, TBuffer&) (in /opt/root/lib/libRIO.so) ==16302== by 0xDD80BB2: TEmulatedCollectionProxy::ReadBuffer(TBuffer&, void*) (in /opt/root/lib/libRIO.so) ==16302== by 0xDD2F508: TBufferFile::ReadFastArray(void*, TClass const*, int, TMemberStreamer*, TClass const*) (in /opt/root/lib/libRIO.so) ==16302== by 0xDE83A98: int TStreamerInfoActions::ReadSTL<&TStreamerInfoActions::ReadSTLMemberWiseSameClass, &TStreamerInfoActions::ReadSTLObjectWiseFastArray>(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) (in /opt/root/lib/libRIO.so) ==16302== by 0xDD2E5C4: TBufferFile::ApplySequence(TStreamerInfoActions::TActionSequence const&, void*) (in /opt/root/lib/libRIO.so) ==16302== by 0xCDE98B0: TBranchElement::ReadLeavesMember(TBuffer&) (in /opt/root/lib/libTree.so) ==16302== by 0xCDE1A0F: TBranch::GetEntry(long long, int) (in /opt/root/lib/libTree.so) ==16302== ==16302== 4,241,664 bytes in 2,104 blocks are possibly lost in loss record 13,789 of 13,790 ==16302== at 0x483BE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==16302== by 0x154CE2CA: clang::ModuleMap::findOrCreateModule(llvm::StringRef, clang::Module*, bool, bool) (in /opt/root/lib/libCling.so) ==16302== by 0x154DBEAF: clang::ModuleMapParser::parseModuleDecl() (in /opt/root/lib/libCling.so) ==16302== by 0x154DBF4F: clang::ModuleMapParser::parseModuleDecl() (in /opt/root/lib/libCling.so) ==16302== by 0x154DCADF: clang::ModuleMapParser::parseModuleMapFile() (in /opt/root/lib/libCling.so) ==16302== by 0x154DCECB: clang::ModuleMap::parseModuleMapFile(clang::FileEntry const*, bool, clang::DirectoryEntry const*, clang::FileID, unsigned int*, clang::SourceLocation) (in /opt/root/lib/libCling.so) ==16302== by 0x154A0D78: clang::HeaderSearch::loadModuleMapFileImpl(clang::FileEntry const*, bool, clang::DirectoryEntry const*, clang::FileID, unsigned int*) (in /opt/root/lib/libCling.so) ==16302== by 0x154A1C58: clang::HeaderSearch::loadModuleMapFile(clang::FileEntry const*, bool, clang::FileID, unsigned int*, llvm::StringRef) (in /opt/root/lib/libCling.so) ==16302== by 0x132DCCD2: (anonymous namespace)::createCIImpl(std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer> >, cling::CompilerOptions const&, char const*, std::unique_ptr<clang::ASTConsumer, std::default_delete<clang::ASTConsumer> >, std::vector<std::shared_ptr<clang::ModuleFileExtension>, std::allocator<std::shared_ptr<clang::ModuleFileExtension> > > const&, bool, bool) (in /opt/root/lib/libCling.so) ==16302== by 0x132DE67F: cling::CIFactory::createCI(llvm::StringRef, cling::InvocationOptions const&, char const*, std::unique_ptr<clang::ASTConsumer, std::default_delete<clang::ASTConsumer> >, std::vector<std::shared_ptr<clang::ModuleFileExtension>, std::allocator<std::shared_ptr<clang::ModuleFileExtension> > > const&) (in /opt/root/lib/libCling.so) ==16302== by 0x13384330: cling::IncrementalParser::IncrementalParser(cling::Interpreter*, char const*, std::vector<std::shared_ptr<clang::ModuleFileExtension>, std::allocator<std::shared_ptr<clang::ModuleFileExtension> > > const&) (in /opt/root/lib/libCling.so) ==16302== by 0x133048C2: cling::Interpreter::Interpreter(int, char const* const*, char const*, std::vector<std::shared_ptr<clang::ModuleFileExtension>, std::allocator<std::shared_ptr<clang::ModuleFileExtension> > > const&, bool, cling::Interpreter const*) (in /opt/root/lib/libCling.so) ==16302== ==16302== LEAK SUMMARY: ==16302== definitely lost: 66,612 bytes in 482 blocks ==16302== indirectly lost: 1,143,862 bytes in 608 blocks ==16302== possibly lost: 12,603,877 bytes in 9,909 blocks ==16302== still reachable: 4,085,245 bytes in 37,394 blocks ==16302== of which reachable via heuristic: ==16302== newarray : 37,648 bytes in 66 blocks ==16302== multipleinheritance: 47,352 bytes in 4 blocks ==16302== suppressed: 22,764,278 bytes in 26,407 blocks ==16302== Reachable blocks (those to which a pointer was found) are not shown. ==16302== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==16302==