abacusnbody.data

compaso_halo_catalog module

See the CompaSO Halo Catalogs page.

pipe_asdf module

See the Unix Pipes: pipe_asdf page.

read_abacus module

This is an interface to read various Abacus file formats, like ASDF, pack9, and RVint.

For particle-oriented access to the data, one can use this interface. For halo-oriented access (e.g. associating halo particles with their host halo), one should use the relevant halo module (like abacusnbody.data.compaso_halo_catalog).

The decoding of the binary formats is generally contained in other modules (e.g. bitpacked); this interface mainly deals with the container formats and high-level logic of file names, Astropy tables, etc.

abacusnbody.data.read_abacus.read_asdf(fn, load=None, colname=None, dtype=<class 'numpy.float32'>, verbose=True, **kwargs)

Read an Abacus ASDF file. The result will be returned in an Astropy table.

Parameters:

• fn (str) – The filename of the ASDF file to load

• load (list of str or None, optional) –

  A list of columns to load. The default (None) is to load columns based on what’s in the file. If the file contains positions and velocities, those will be loaded; if it contains PIDs, those will be loaded.

  The list of fields that can be specified is: 'pos', 'vel', 'pid', 'lagr_pos', 'tagged', 'density', 'lagr_idx', 'aux'

  All except pos & vel are PID-derived fields (see abacusnbody.data.bitpacked.unpack_pids())

• colname (str or None, optional) – The internal column name in the ASDF file to load. Probably one of 'rvint', 'packedpid', 'pid', or 'pack9'. In most cases, the name can be automatically detected, which is the default behavior (None).

• dtype (np.dtype, optional) – The precision in which to unpack any floating point arrays. Default: np.float32

• verbose (bool, optional) – Print informational messages. Default: True

Returns:

table – A table whose columns contain the particle data from the ASDF file. The meta field of the table contains the header.

Return type:

astropy.Table

bitpacked module

A collection of routines related to various Abacus bitpacked formats, like RVint and encoding of information in the PIDs.

Most users will not use this module directly, but will instead use abacusnbody.data.compaso_halo_catalog or abacusnbody.data.read_abacus.read_asdf().

abacusnbody.data.bitpacked.unpack_rvint(intdata, boxsize, float_dtype=<class 'numpy.float32'>, posout=None, velout=None)

Unpack rvint data into pos and vel.

Parameters:

• intdata (ndarray of dtype np.int32) – The rvint data

• boxsize (float) – The box size, used to scale the positions

• float_dtype (np.dtype, optional) – The precision in which to store the unpacked values. Default: np.float32

• posout (ndarray, None, or False; optional) – The array in which to store the unpacked positions. None can be given (the default), in which case an array is constructed and retured as the first return value. False can be given, in which case the positions are not unpacked.

• velout (optional) – Same as posout, but for the velocities

Returns:

pos,vel – A tuple of the unpacked position and velocity arrays, or the number of unpacked particles if an output array was given.

Return type:

tuple

abacusnbody.data.bitpacked.unpack_pids(packed, box=None, ppd=None, pid=False, lagr_pos=False, tagged=False, density=False, lagr_idx=False, float_dtype=<class 'numpy.float32'>)

Extract fields from bit-packed PIDs. The PID (really, the 64-bit aux field) enocdes the particle ID, the Lagrangian index (and therefore position), the density, and the L2 tagged field.

Parameters:

• packed (array-like of np.uint64, shape (N,)) – The bit-packed PID (i.e. the aux field)

• box (float, optional) – The box size, needed only for lagr_pos

• ppd (int, optional) – The particles-per-dimension, needed only for lagr_pos

• pid (bool, optional) –

  Whether to unpack and return the unique particle ID.

  Loaded as a np.int64 array of shape (N,).

• lagr_idx (bool, optional) –

  Whether to unpack and return the Lagrangian index, which is the (i,j,k) integer coordinates of the particle in the cubic lattice used in Abacus pre-initial conditions. The lagr_pos field will automatically convert this index to a position.

  Loaded as a np.int16 array of shape (N,3).

• lagr_pos (bool, optional) –

  Whether to unpack and return the Lagrangian position of the particles, based on the Lagrangian index (lagr_idx).

  Loaded as array of type dtype and shape (N,3).

• tagged (bool, optional) –

  Whether to unpack and return the CompaSO L2 tagged bit of the particles— whether the particle was ever part of an L2 group (i.e. halo core).

  Loaded as a np.bool8 array of shape (N,).

• density (bool, optional) –

  Whether to unpack and return the local density estimate, in units of mean density.

  Loaded as a array of type dtype and shape (N,).

• float_dtype (np.dtype, optional) – The dtype in which to store float arrays. Default: np.float32

Returns:

unpacked_arrays – A dictionary of all fields that were unpacked

Return type:

dict of ndarray

asdf module

This module contains the ASDF extensions that allow the asdf Python package to read Abacus ASDF files that use Blosc compression internally.

There are two classes here: an Extension subclass, and a Compressor subclass. The Extension is registered with ASDF via a setuptools “entry point” in setup.py. It contains the reference to the Compressor subclass that knows how to handle Blosc compression.

abacusnbody.data.asdf.set_nthreads(nthreads)

class abacusnbody.data.asdf.BloscCompressor

Bases: Compressor

An implementation of Blosc compression, as used by Abacus.

property label

The string labels in the binary block headers that indicate Blosc compression

compress(data, **kwargs)

Useful compression kwargs: nthreads compression_block_size blosc_block_size shuffle typesize cname clevel

decompress(blocks, out, **kwargs)

Useful decompression kwargs: nthreads

class abacusnbody.data.asdf.AbacusExtension

Bases: Extension

An ASDF Extension that deals with Abacus types and formats. Currently only implements Blosc compression.

property extension_uri

Get the URI of the extension to the ASDF Standard implemented by this class. Note that this may not uniquely identify the class itself.

Return type:

str

property compressors

Return the Compressors implemented in this extension

pack9 module

Unpack pack9 particle data, which encodes the pos + vel in 9 bytes, and then the PID + aux in another 8 bytes in a separate file. The 9-byte part is handled by this module; the 8-byte (PID) part is handled by bitpacked.unpack_pids().

Most users will not use this module directly, but will instead use the abacusnbody.data.read_abacus.read_asdf() function.

abacusnbody.data.pack9.unpack_pack9(data, boxsize, velzspace_to_kms, float_dtype=<class 'numpy.float32'>, posout=None, velout=None)