Pfthb

For Unixes (Linux, Mac OS X, Solaris) you could also use the getrusage() function from the standard library module resource. The resulting object has the attribute ru_maxrss, which gives peak memory usage for the calling process:

>>> resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
2656 # peak memory usage (bytes on OS X, kilobytes on Linux)

The Python docs aren't clear on what the units are exactly, but the Mac OS X man page for getrusage(2) describes the units as bytes. The Linux man page isn't clear, but it seems to be equivalent to the information from /proc/self/status, which is in kilobytes.

The getrusage() function can also be given resource.RUSAGE_CHILDREN to get the usage for child processes, and (on some systems) resource.RUSAGE_BOTH for total (self and child) process usage.

resource is a standard library module.

If you only care about Linux, you can just check the /proc/self/status file as described in a similar question.

On Windows, you can use WMI (home page, cheeseshop):

def memory():
 import os
 from wmi import WMI
 w = WMI('.')
 result = w.query("SELECT WorkingSet FROM Win32_PerfRawData_PerfProc_Process WHERE IDProcess=%d" % os.getpid())
 return int(result[0].WorkingSet)

On Linux (from python cookbook http://code.activestate.com/recipes/286222/:

import os
_proc_status = '/proc/%d/status' % os.getpid()

_scale = 'kB': 1024.0, 'mB': 1024.0*1024.0,
 'KB': 1024.0, 'MB': 1024.0*1024.0

def _VmB(VmKey):
 '''Private.
 '''
 global _proc_status, _scale
 # get pseudo file /proc/<pid>/status
 try:
 t = open(_proc_status)
 v = t.read()
 t.close()
 except:
 return 0.0 # non-Linux?
 # get VmKey line e.g. 'VmRSS: 9999 kBn ...'
 i = v.index(VmKey)
 v = v[i:].split(None, 3) # whitespace
 if len(v) < 3:
 return 0.0 # invalid format?
 # convert Vm value to bytes
 return float(v[1]) * _scale[v[2]]


def memory(since=0.0):
 '''Return memory usage in bytes.
 '''
 return _VmB('VmSize:') - since


def resident(since=0.0):
 '''Return resident memory usage in bytes.
 '''
 return _VmB('VmRSS:') - since


def stacksize(since=0.0):
 '''Return stack size in bytes.
 '''
 return _VmB('VmStk:') - since

On unix, you can use the ps tool to monitor it:

$ ps u -p 1347 | awk 'sum=sum+$6; END print sum/1024'

where 1347 is some process id. Also, the result is in MB.

Heapy (and friends) may be what you're looking for.

Also, caches typically have a fixed upper limit on their size to solve the sort of problem you're talking about. For instance, check out this LRU cache decorator.

I like it, thank you for @bayer. I get a specific process count tool, now.

# Megabyte.
$ ps aux | grep python | awk 'sum=sum+$6; END print sum/1024 " MB"'
87.9492 MB

# Byte.
$ ps aux | grep python | awk 'sum=sum+$6; END print sum " KB"'
90064 KB

Attach my process list.

$ ps aux | grep python
root 943 0.0 0.1 53252 9524 ? Ss Aug19 52:01 /usr/bin/python /usr/local/bin/beaver -c /etc/beaver/beaver.conf -l /var/log/beaver.log -P /var/run/beaver.pid
root 950 0.6 0.4 299680 34220 ? Sl Aug19 568:52 /usr/bin/python /usr/local/bin/beaver -c /etc/beaver/beaver.conf -l /var/log/beaver.log -P /var/run/beaver.pid
root 3803 0.2 0.4 315692 36576 ? S 12:43 0:54 /usr/bin/python /usr/local/bin/beaver -c /etc/beaver/beaver.conf -l /var/log/beaver.log -P /var/run/beaver.pid
jonny 23325 0.0 0.1 47460 9076 pts/0 S+ 17:40 0:00 python
jonny 24651 0.0 0.0 13076 924 pts/4 S+ 18:06 0:00 grep python

Reference

• memory - Linux: find out what process is using all the RAM? - Super User


• Total memory used by Python process? - Stack Overflow


• linux - ps aux output meaning - Super User

Below is my function decorator which allows to track how much memory this process consumed before the function call, how much memory it uses after the function call, and how long the function is executed.

import time
import os
import psutil


def elapsed_since(start):
 return time.strftime("%H:%M:%S", time.gmtime(time.time() - start))


def get_process_memory():
 process = psutil.Process(os.getpid())
 return process.get_memory_info().rss


def track(func):
 def wrapper(*args, **kwargs):
 mem_before = get_process_memory()
 start = time.time()
 result = func(*args, **kwargs)
 elapsed_time = elapsed_since(start)
 mem_after = get_process_memory()
 print(": memory before: :,, after: :,, consumed: :,; exec time: ".format(
 func.__name__,
 mem_before, mem_after, mem_after - mem_before,
 elapsed_time))
 return result
 return wrapper

So, when you have some function decorated with it

from utils import track

@track
def list_create(n):
 print("inside list create")
 x = [1] * n
 return x

You will be able to see this output:

inside list create
list_create: memory before: 45,928,448, after: 46,211,072, consumed: 282,624; exec time: 00:00:00

import os, win32api, win32con, win32process
han = win32api.OpenProcess(win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ, 0, os.getpid())
process_memory = int(win32process.GetProcessMemoryInfo(han)['WorkingSetSize'])

Current memory usage of the current process on Linux, for Python 2, Python 3, and pypy, without any imports:

def getCurrentMemoryUsage():
 ''' Memory usage in kB '''

 with open('/proc/self/status') as f:
 memusage = f.read().split('VmRSS:')[1].split('n')[0][:-3]

 return int(memusage.strip())

Tested on Linux 4.4 and 4.9, but even an early Linux version should work.

Looking in man proc and searching for the info on the /proc/$PID/status file, it mentions minimum versions for some fields (like Linux 2.6.10 for "VmPTE"), but the "VmRSS" field (which I use here) has no such mention. Therefore I assume it has been in there since an early version.

Using sh and os to get into python bayer's answer.

float(sh.awk(sh.ps('u','-p',os.getpid()),'sum=sum+$6; END print sum/1024'))

Answer is in megabytes.

For Python 3.6 and psutil 5.4.5 it is easier to use memory_percent() function listed here.

import os
import psutil
process = psutil.Process(os.getpid())
print(process.memory_percent())

Even easier to use than /proc/self/status: /proc/self/statm. It's just a space delimited list of several statistics. I haven't been able to tell if both files are always present.

/proc/[pid]/statm

Provides information about memory usage, measured in pages.
The columns are:

• size (1) total program size
  (same as VmSize in /proc/[pid]/status)


• resident (2) resident set size
  (same as VmRSS in /proc/[pid]/status)


• shared (3) number of resident shared pages (i.e., backed by a file)
  (same as RssFile+RssShmem in /proc/[pid]/status)


• text (4) text (code)


• lib (5) library (unused since Linux 2.6; always 0)


• data (6) data + stack


• dt (7) dirty pages (unused since Linux 2.6; always 0)

Here's a simple example:

from pathlib import Path
from resource import getpagesize


def get_resident_set_size():
 # Columns are: size resident shared text lib data dt
 statm = Path('/proc/self/statm').read_text()
 fields = statm.split()
 return int(fields[1]) * getpagesize()


data = 
start_memory = get_resident_set_size()
for _ in range(10):
 data.append('X' * 100000)
 print(get_resident_set_size() - start_memory)

That produces a list that looks something like this:

0
0
368640
368640
368640
638976
638976
909312
909312
909312

You can see that it jumps by about 300,000 bytes after roughly 3 allocations of 100,000 bytes.