Changelog

memray 1.14.0 (2024-09-09)

Features

  • Wheels are now published for Python 3.13, though we don’t yet support free-threaded (no GIL) builds. (#658)

Bug Fixes

  • Fix a lock ordering deadlock in libc between a Memray lock and a lock internal to dlopen. (#549)

  • Ensure flame graphs stay in flame mode when the user has selected it. Previously clicking the “Hide Irrelevant Frames” or “Hide Import System Frames” checkboxes would switch the flame graph back to icicle mode. (#656)

  • Fix a race condition that was able to cause strange exception messages if two different threads tried to initialize Memray tracking at once. (#667)

memray 1.13.4 (2024-07-18)

Bug Fixes

  • A backwards-incompatible change released in Textual 0.73 caused the memray tree reporter to start with no nodes expanded. This release adds a workaround to restore the old behavior of expanding the first child of each node. (#648)

memray 1.13.3 (2024-07-02)

Bug Fixes

  • Fix a bug that could result in truncated reports for applications that fork without calling PyOS_BeforeFork(), including by using multiprocessing with the “spawn” start method (the default on macOS). (#644)

memray 1.13.2 (2024-06-27)

Bug Fixes

  • Fix a bug that could in rare circumstances result in a stack overflow while processing native mode stacks. (#639)

Miscellaneous

  • Upgrade our vendored copy of libbacktrace, used for reporting native stacks, to the latest version. (#639)

memray 1.13.1 (2024-06-23)

Bug Fixes

  • Fix a deadlock that could occur on some Linux systems when resolving debug information using debuginfod. (#634)

memray 1.13.0 (2024-06-18)

Features

  • Add a tutorial to the Memray documentation. (#590)

  • Include the thread name in the live TUI. (#562)

  • Capture the name attribute of Python threading.Thread objects. (#562)

  • Allow using Ctrl+Z to suspend memray tree and the live mode TUI. (#581)

  • Add a button in the live-mode TUI to show allocations from all threads at once. (#589)

  • Vendor libdebuginfod into our Linux wheels, so that debuginfod integration can be used without any dependency on system-installed libraries. (#592)

Bug Fixes

  • Fix dynamic toggling between descriptions like “Pause” vs “Unpause” or “Show” vs “Hide” in the footer of the live-mode TUI and tree reporter. This was broken by changes introduced in Textual 0.61 (and again by Textual 0.63). (#597)

  • Correctly localize the start and end time in the “Stats” modal when an HTML report was generated on a different machine than the one it is being displayed on. (#611)

  • Fix a crash in old macOS versions (<11.0) due to the inavailability of some linker cache APIs. (#615)

  • Fix reporting of “Own Memory” in the live and summary reporters. A bug in our summation caused us to undercount functions’ direct allocations. (#617)

Miscellaneous

  • Builds from source now work for Python 3.13. Wheels are not yet published for 3.13 because it is not yet ABI stable. (#622)

  • Link our Linux wheels against the latest version of elfutils. (#592)

memray 1.12.0 (2024-03-07)

Features

  • Allow --temporal and --max-memory-records to be used with our Jupyter magic. (#538)

  • Automatically use aggregated capture files for the Jupyter magic whenever possible, reducing the amount of disk space needed for temporary files. (#538)

  • Expose the main thread id in the FileReader’s metadata attribute. (#560)

Bug Fixes

  • Fix a bug that was causing dlopen to not load shared libraries that have an RPATH/RUNPATH set. (#525)

  • Fix a bug where the tree reporter would fail to populate the code pane with relevant lines if the line where the allocation occurred was too near the start of the file. (#544)

  • Fix a bug causing the first entry of sys.path to be erroneously overwritten by memray run when the Python interpreter was launched with the -I or -P flag, or when the PYTHONSAFEPATH environment variable was set. (#552)

memray 1.11.0 (2023-12-04)

Features

  • Migrate the live TUI to Textual. This provides a greatly improved user experience, including the ability to scroll to view rows that don’t fit on the screen. (#274)

  • Add a new documentation page to serve as an overview of memory concepts, to help users better interpret the memory profiles provided by Memray. (#496)

  • Where possible, leverage pkg-config when building the extension from source, picking up appropriate compiler and linker flags automatically. (#498)

  • Port the tree reporter to be an interactive Textual App. (#499)

Bug Fixes

  • Fixed a bug that caused memray attach to fail with newer LLDB versions, including on macOS Sonoma. (#490)

  • Limit the number of memory records displayed in reporters by default. This will help displaying flamegraphs for long capture sessions. (#491)

  • When generating a --leaks flamegraph, don’t show a warning that the pymalloc allocator is in use if --trace-python-allocators was used when generating the capture file. (#492)

  • Ensure that we update our terminal progress bars to 100% when processing finishes. (#494)

memray 1.10.0 (2023-10-05)

Features

  • Add support for Inverted flame graphs. In an inverted flame graph, the roots are the functions that allocated memory, and the children of any given node represent the percentage of that node’s allocations that can be attributed to a particular caller. The inverted flame graph is very helpful in analyzing where memory is being spent in aggregate. You can generate one by passing the --inverted flag to memray flamegraph. (#439)

  • memray attach now supports --aggregate to produce aggregated capture files. (#455)

  • memray attach has been enhanced to allow tracking for only a set period of time. (#458)

  • A new memray detach command allows you to manually deactivate tracking that was started by a previous call to memray attach. (#458)

  • Python 3.12 is now supported. (#474)

Bug Fixes

  • Update memray attach on Linux to prefer GDB over LLDB for injecting itself into the process being attached to. We’ve had several reports of problems with the Linux LLDB, and hope this change will help give Linux users a better experience by default. You can still explicitly use LLDB on Linux even when GDB is detected by running memray attach --method=lldb. (#449)

  • Fix a memory leak in Memray itself when many different capture files are opened by a single Memray process and native stacks are being reported. This issue primarily affected pytest-memray. (#473)

  • Fix a crash in MacOS Sonoma when using system Framework libraries, like when using the pyobjc library. (#477)

memray 1.9.1 (2023-08-01)

Bug Fixes

  • Fix an issue that stopped Memray’s experimental support for greenlet from working with versions of the greenlet module older than 1.0. (#432)

  • Fix a bug leading to a deadlock when Memray is used to profile an application that uses the jemalloc implementation of malloc. (#433)

  • Fix a bug causing the summary reporter to generate empty reports. (#435)

memray 1.9.0 (2023-07-28)

Features

  • Allow to report the current version of Memray via a --version/-V command line parameter (#420)

  • Add pause/unpause keybindings to the live reporter that allows the user to pause the live reporter to analyse the current results without pausing the running program (#418)

Bug Fixes

  • Support building with Cython 3 (#425)

memray 1.8.1 (2023-06-20)

Features

  • When the high water mark being shown by a temporal flame graph is before the first memory snapshot or after the last one, tell the user so by highlighting a region beyond the end of the memory usage plot. (#399)

Bug Fixes

  • Prevent a totally empty memory plot from being shown on flame graphs when the tracked process completes before any periodic memory snapshots are captured. (#399)

  • Fix a bug that prevented the temporal high water mark flame graph from showing the flame graph of a high water mark that occurred after the final periodic memory snapshot was captured. (#399)

  • Fix a bug that prevented Memray from intercepting functions in shared objects that are part of the dyld shared cache in macOS Ventura. (#401)

memray 1.8.0 (2023-06-09)

Features

  • Allow memray stats to output a JSON report via --json flag. (#377)

  • We now publish x86-64 musllinux_1_1 wheels, compatible with Alpine Linux. (#379)

  • We now support Temporal Flame Graphs, which provide an exciting new way of analyzing your process’s memory usage over time. (#391)

Bug Fixes

  • Fix a bug where a non-import call on the same line as an import statement would be hidden by the “Hide Import System Frames” checkbox of a flame graph. (#329)

  • Fixed a bug that was hitting an assert when constructing hybrid stack frames in Python 3.11 when no eval symbols are available. (#334)

  • Change the font color used by the %%memray_flamegraph Jupyter magic’s progress updates for better contrast on the JupyterLab dark theme. (#344)

  • Fix a bug that could result in a deadlock when tracking a process linked against an old version of musl libc. (#379)

memray 1.7.0 (2023-02-21)

Features

  • memray run now supports --aggregate to produce aggregated capture files, which can be much smaller but aren’t able to be used for generating every type of report. (#277)

  • Add integration with debuginfod to automatically download debug information for binaries if it is available. (#308)

  • Flame graphs produced by memray flamegraph are now around 85% smaller. (#314)

Bug Fixes

  • memray run --live and memray run --live-remote silently dropped the --trace-python-allocators flag. This has been fixed, and the flag is now properly propagated from the CLI to the tracker. (#283)

  • Fix a bug that was causing Memray to crash when the Tracker is being destroyed and some other thread is still registering allocations or deallocations (#289)

  • Work around a bug in GDB versions before 10.1 that could cause memray attach to fail. (#310)

  • Work around a bug in LLDB on Linux that could cause memray attach to hang. (#311)

memray 1.6.0 (2023-01-17)

Features

  • Speed up native allocation tracking by up to 45% (#294)

Bug Fixes

  • memray run --live and memray run --live-remote silently dropped the --trace-python-allocators flag. This has been fixed, and the flag is now properly propagated from the CLI to the tracker. (#283)

  • Fix a bug that was causing Memray to crash when the Tracker is being destroyed and some other thread is still registering allocations or deallocations (#289)

Memray 1.5.0 (2022-12-09)

Features

  • Memray is now fully supported on macOS, and the warnings that macOS support is experimental have been dropped. (#194)

  • Add a checkbox to flamegraphs that allows hiding frames from the import system (#261)

  • memray attach can be used to attach to a running process (#266)

  • Consider frames from the import system as “irrelevant” in the generated flamegraphs. (#268)

memray 1.4.1 (2022-11-11)

Bug Fixes

  • Fix a crash that can happen when two different threads try to register frames at the same time without the GIL held. (#251)

memray 1.4.0 (2022-10-31)

Features

  • Add a new transform subcomand that allows transforming Memray capture files into output files compatible with other tools. We’re starting by supporting conversions to the gprof2dot format, which allows producing graph-like reports when combined with graphviz. (#200)

  • Added a new --temporary-allocations option to the flamegraph, table, tree, and summary reporters for showing the temporary allocations instead of the high water mark ones. (#201)

  • When the greenlet module is in use, also assign a distinct thread ID to each greenlet. Greenlets aren’t threads, but they are distinct threads of execution within a single process, with distinct stacks, so assigning different thread IDs to each makes it easier to interpret reports where greenlet was used. (#209)

  • Use a monotonic counter to generate thread IDs, rather than using the pthread ID. Those pthread IDs can be reused, making it difficult to tell what thread performed an allocation. (#209)

  • Print a warning when we detect that the Python interpreter was built without debug information or without symbols, letting the user know in advance that these conditions may result in incorrect stack traces or missing filenames and line numbers. (#211)

  • A new %%memray_flamegraph Jupyter cell magic is provided by %load_ext memray, and can be used to memory profile code directly in a Jupyter notebook. (#237)

  • Add csv as a possible target format for memray transform, producing a report of all of the allocations that made up the process’s high water mark of allocated memory. This CSV file can then be loaded and analyzed using libraries like pandas. (#241)

Deprecations and Removals

  • Up until now, if the program being profiled included a Cython module built with profiling support enabled, those Cython functions would show up in our Python call stacks. This was rarely useful in practice, as most Cython libraries aren’t distributed with profiling support enabled, and supporting this had a surprisingly high maintenance cost. We’ve removed this integration, so you’ll need to use --native mode to see inside of Cython modules. We are not considering this a backwards-incompatible change, since it does not affect any of our public interfaces (though it could affect the contents of reports generated by Memray). (#206)

Bug Fixes

  • Fix a bug that caused incorrect --native mode stacks on Python 3.11 for allocations performed directly by the interpreter’s eval loop. (#209)

  • Fix a crash when an extension module terminates the program using non-Python APIs under tracking. (#228)

memray 1.3.1 (2022-08-30)

Bug Fixes

  • Prevent a crash that could occur when forked processes that have been under tracking without follow_fork=True remove the profiling function with pending frames needed to be flushed to the results file. (#196)

memray 1.3.0 (2022-08-18)

Features

  • We now capture Python stacks for allocations made by threads that existed before the Memray tracker was started. (#130)

  • Add support for Python 3.11 (#138)

  • Add support for MacOS. (#174)

  • Add experimental support for Greenlet. (#185)

Bug Fixes

  • Prevent a crash that could occur if the Memray API was used to stop and later restart tracking while another thread was running Python code. (#152)

  • Prevent a use-after-free bug that could result in a crash if sys.setprofile() was called while Memray was tracking. Now if sys.setprofile() is called, all future allocations on that thread will report unknown Python stacks, instead of potentially incorrect stacks. (#176)

Memray 1.2.0 (2022-07-11)

Features

  • Add a progress bar indicator to the record processing phases in the different reporters so users can have an approximate idea of how much time processing the result files will take. (#111)

  • The memray stats reporter is now up to 50% faster, and its output is easier to interpret because it now processes all allocations by default. (#136)

  • Add a line showing the heap size over time to the memory plot in the html-based reporters (which already showed the resident size over time). (#142)

Deprecations and Removals

  • Remove the --include-all-allocations / -a argument to the memray stats reporter. Previously this was too slow to be used by default, but now that it has been sped up, it doesn’t make sense to use anything else. The old default behavior of only processing allocations that made up the high water mark of the application’s memory usage was confusing and misleading. (#136)

Bug Fixes

  • Fix a crash with SIGBUS when the file system fills up while memray run is writing a capture file. (#117)

  • Recognize when a capture file has been truncated (most likely because the tracked process was killed unexpectedly) and ignore any incomplete record at the end of the file. (#129)

  • Fix the histogram used by the memray stats reporter to choose sane bin sizes when all captured allocations are the same size. (#133)

  • Fix the aggregation by location at the bottom of the memray stats report when the --include-all-allocations option is used. (#134)

  • Fix a bug causing deallocations with free and munmap to be included in the reported “Total allocations” count of memray stats --include-all-allocations. (#136)

  • Fix the two “largest allocating locations” sections in the memray stats report to actually aggregate by location. Previously they were aggregating by distinct stacks, so if two different paths hit the same line of code, it would be counted separately instead of together. (#136)

  • Fix a bug causing memory freed by munmap to be incorrectly added into the reported “Total memory allocated” of memray stats --include-all-allocations. (#136)

  • Exclude PYMALLOC_FREE from the allocator type distribution (other deallocators were already being ignored, but this recently added one was missed). (#136)

  • Fix the memray stats histogram to be based on the actual sizes of all allocations. Previously it only saw the sizes after a rollup by stack had already been performed, so it was binning allocation sizes that had already been summed. (#136)

  • Fixed a bug where aggregating native call stacks could give misleading results on aarch64 under some circumstances. (#141)

  • Fix a bug that made memray run --live -c fail if the command to run contained double quotes. (#147)

  • Ensure our TUI isn’t displaying stale data by periodically flushing the latest available data from the tracker (rather than only flushing when a buffer fills up). (#147)

  • Fix the handling of the thread switch commands in the live mode TUI before the first allocation has been seen. (#147)

memray 1.1.0 (2022-05-16)

Features

  • Finalize and document the Memray tracking API. (#42)

  • Ensure that wheels built by make dist are reproducible (so that running the build twice produces identical artifacts). (#47)

  • Reduce the size of the memray run capture file by around 20% by using a more efficient encoding for which allocator was used to perform a given allocation and whether we captured a native stack for that allocation. (#52)

  • Support memray run -c "..." to profile an in-line script provided on the command line. (#61)

  • The capture files produced by memray run are now around 90% smaller thanks to a more efficient encoding scheme for the binary files. (#67)

  • Add support for Alpine Linux and musl libc. (#75)

  • Capture allocations made through the C99 aligned_alloc function. (#79)

  • By default the capture file will now be compressed using LZ4 after tracking completes. This temporarily requires extra disk space while the compression runs, but results in roughly 75% less disk space required in the end. Compression can be disabled with --no-compress. (#82)

  • Speed up tracking by around 5% by building with link-time optimization (LTO). (#91)

  • Add a new --trace-python-allocators option to memray run that allows tracking all allocations made using the Python allocators. This will result in bigger output files and slower profiling but it allows getting insights about all of the interpreter’s memory allocations. (#92)

Bug Fixes

  • Previously we attempted to read all allocation records into memory when processing a capture file in our reporters. This could fail on large files, so now we process the file in a streaming fashion instead. (#62)

  • Make memray run perform the same modifications to sys.path as the interpreter itself would when running a script. (#86)

  • Fixed a bug in the stats reporter that could result in the largest allocations being omitted from the histogram. (#95)

  • Fixed a bug that caused Memray reporters to display incorrect stacks when Native tracking was enabled and native allocations from different locations occurred underneath the same Python stack. (#96)

Miscellaneous

  • Support the latest versions of Rich (previously we pinned to an old version due to some formatting changes in more recent versions). (#98)

memray 1.0.3 (2022-04-21)

Features

  • Add memray as a command line entry point. (#20)

memray 1.0.2 (2022-04-12)

Features

  • Add publishing of ManyLinux2010 wheels for 64 and 32 bit systems. (#2)

Bug Fixes

  • Fix 32 bit builds. (#2)

memray 1.0.0 (2022-04-09)

  • Initial release.