Measurement Sets and Calibration Tables

Everything that knows the Measurement Set format. Named for the format rather than generically, so that a second input format becomes a sibling module with its own name.

Reading declarations rather than assuming them

An MS records several properties that are easy to assume and expensive to get wrong. TABASCAL reads them from the file:

Correlations. The POLARIZATION subtable’s CORR_TYPE lists which correlations the MS actually holds, as CASA Stokes codes. resolve_correlation() matches the configured data.corr against it by identity, not by position, so yy selects YY whether the MS holds all four correlations or only that one. A fixed {xx: 0, xy: 1, yx: 2, yy: 3} table only works for a full four-correlation MS: a single-correlation MS has a length-1 correlation axis whatever polarisation it holds, and a two-correlation (XX, YY) MS puts YY at index 1. Requesting a correlation the MS does not hold is an error naming what it does hold, rather than an index error or a silent read of the wrong polarisation.

Time scale. The TIME column’s MEASINFO record declares the scale its values are on, almost always UTC. read_time_scale() returns it, and it is carried in the read_ms result as time_scale. The scales differ by enough to matter — reading a UTC epoch as TAI shifts it by the accumulated leap seconds, 37 s since 2017, or roughly 285 km along a LEO satellite’s ground track — and none of these mismatches raise.

The declaration is honoured by normalising to UTC once, in read_ms: to_utc_jd() moves the declared Julian Dates onto UTC and the result is returned as times_jd. Everything past the reader works on that — skyfield through skyfield_time()’s default, sgp4jax’s propagator, which has no scale concept to be told otherwise, and the TLE epoch checks — so a single conversion covers all of them and no scale argument is threaded through the trajectory maths. A UTC-declared MS goes through no arithmetic at all and reads bit-identically.

An epoch reference TABASCAL cannot place on a time line — a sidereal angle such as GAST, or a relativistic scale skyfield offers no constructor for — stops the read, before the visibilities are touched, rather than being guessed at.

The preflight observation-epoch helper — which runs before the reader, and which every TLE age comparison and nearest-record decision is measured from — reads the same MEASINFO record through casacore and normalises the same way, so it lands on the same instant. That keeps it physically right and keeps check_epoch_agreement comparing like with like, both sides on UTC.

times_mjd stays as declared beside times_jd: it is the MS’s own TIME column in days — the numbers the file stores, converted in unit only — and ms_integration_times_mjd() reports that same column the same way, so the two stay comparable. It is kept as read rather than recovered from times_jd, which loses ~1e-10 days on the round trip. Nothing writes it back to the MS: the results writer leaves the original TIME column untouched.

Time unit. The same column’s QuantumUnits keyword declares whether its values are seconds or days; TABASCAL works in MJD days. read_time_unit() returns the declaration and times_to_mjd() applies it, falling back — for the columns that carry no declaration — on the magnitude of the times: an MJD day number is at most ~1e5 in any plausible observing era against ~1e9 for the same instant in seconds, and no observation falls between. What is compared is the median of the finite magnitudes, so no single entry decides the column — a row casacore added and never filled leaves TIME at zero, and the smallest magnitude would read a column of seconds as days on the strength of it. The threshold is strict: a magnitude of exactly 1e5 reads as days.

The spacing of consecutive samples used to decide this instead, on the reasoning that an integration is seconds long and so a gap above half a day could only be seconds. It could not. The threshold was strict, so an integration of exactly 0.5 s — a common correlator dump time — read as days, and so did anything shorter; that is issue #208, which showed up as an OverflowError out of the preflight TLE epoch check, times of ~5e9 having been taken for day numbers. And in the other direction a day-numbered column stepping past the half-day threshold read as seconds: MJD 60676 came back as MJD 0.7. The rule compared the two smallest distinct times rather than a representative cadence, so this needed every sample to be at least half a day from the next — a column carrying one row per day, not merely an observation spread over several.

Magnitude parts every column the spacing rule parted correctly inside the era the threshold already assumes, so the rule is gone rather than repaired: for a spacing test to decide anything magnitude does not, a column stored in seconds would need a typical |TIME| of 1e5 or less, putting the observation within 1.16 days of the MJD epoch of 1858-11-17. Outside the era bound — a day-numbered column past 2132, whose |TIME| exceeds 1e5 — spacing was the better of the two, but a heuristic whose constant is an era bound has conceded that case already.

read_ms(), the results writer’s observation grid and the preflight observation-epoch helper all convert here, so the heuristic cannot classify one MS two ways — including for an MS whose timestep blocks do not ascend, since a median does not depend on the order its values arrive in. The preflight helper deduplicates and the other two do not, but it converts before it deduplicates, and on any MS ms_layout() accepts the three are handed the same multiset anyway: n_time is the number of distinct times and every block must hold one constant time, so the reader’s slice holds nothing for np.unique to remove. Their scopes still differ — the preflight helper reads the whole main table and the reader one partition — which weighing the times by frequency is what makes survivable.

A declared unit cannot part them either. The preflight helper reads QuantumUnits from the same getcolkeywords call it takes MEASINFO from, so all three converters honour a declaration and the heuristic is what is left for an MS carrying none. It deliberately did not, once, so that both paths would share the heuristic — but the reader honours a declaration, so leaving it unread was what created the divergence rather than what closed it: an MS declaring seconds while storing day numbers was read on the declaration by the run and on the magnitudes by the TLE age checks.

CASA calibration tables

Gains are exchanged in CASA’s own format rather than as ad-hoc .npz files, so that standard tooling — applycal, CARAcal, stimela — can consume what TABASCAL solves for. write_caltable() emits a B Jones table laid out exactly as casatasks.gaincal emits one: one row per (time, antenna), time-major, with CPARAM of shape (n_chan, n_pol). B Jones rather than G Jones because the gains are frequency dependent, which a scalar G table cannot carry.

CASA identifies a caltable by its table INFO record, not by its keywords: without type='Calibration' applycal rejects the table outright. The MS’s ANTENNA, FIELD, SPECTRAL_WINDOW, OBSERVATION and HISTORY subtables are copied in beside it, as CASA does, which is what lets read_caltable() return the channel frequencies the gains belong to without being handed the MS again.

The convention is CASA’s: V_obs = g_p conj(g_q) V_true, so calibrating divides that out and the noise follows the data, sigma_cal = sigma / |g_p conj(g_q)|. apply_gains_to_data() is that one statement in code.

Extra keywords are written beside the table’s own, for what the solver knows and the format has no field for: TABASCAL records the correlation it fitted as FittedCorr, since a single-solution table otherwise cannot say, and applying an xx solution to yx data is a silent mistake. The names the table needs for itself — the four CASA identifies it by, and the one per subtable — are refused rather than overwritten, and so are values casacore cannot be relied on to encode: a keyword value is a string, bool, int or float, or a non-empty list of one single one of those. read_caltable() returns the caller’s keywords, and only those, so a round trip needs no casacore.

time_ref is the epoch reference the TIME column declares, and it must be the MS’s own. The times are a copy of the MS’s column and nothing shifts them, so a table declaring UTC over a TAI-declared observation has moved every timestamp by the accumulated leap seconds for anything that reads the declaration. It is validated against TIME_SCALES, and read back as time_ref.

remove_caltable() is the other end: it deletes a table whose solution has been superseded — a rerun that fits no gains has none to overwrite the previous one with, and a stale table under the current name reads as the current calibration. It applies the same overlap guard as the writer, and the same test of what it is about to delete.

What may be deleted

Both calls that remove something — remove_caltable, and write_caltable clearing an existing output under overwrite=True — first ask whether the path holds a table this module could have written. Three checks, each catching what the others miss: casacore’s marker files, which a caller’s own directory and an ordinary file do not have; an INFO record declaring Type = Calibration, which is the only thing separating a caltable from a Measurement Set, since an MS carries the same markers; and tableexists, casacore’s read-only structural check, which catches a directory dressed up with hand-written marker files and a table.dat that has been truncated or replaced.

That is a check on the format and not a guarantee of integrity — a table that opens cleanly can still hold a solution that is wrong, or for another observation — but it does guarantee that what is removed was a casacore calibration table rather than a caller’s data. Anything else at the path is a ValueError from the writer and a False from the remover; a damaged table is refused on the same rule and has to be cleared by hand. This tightens the original overwrite contract, which removed the destination on sight: every legitimate overwrite target is a previous solution, so anything else there is a path pointing somewhere the caller did not mean, and the cost of reading it the other way is a deletion that cannot be undone.

The solution TABASCAL fits is exported this way after every run; see Output from TABASCAL.

A gain that is zero or non-finite carries no solution, and both halves of that are written: FLAG is set and CPARAM is NaN, so a reader going by the flag and one going by the value reach the same conclusion. Calibrating with one gives NaN rather than an infinity — every kind of dead gain arrives as the same NaN, so a caller flagging on isnan catches all of them.

Scope: one spectral window, one correlation

That is what TABASCAL fits, and it is checked rather than assumed. Every row is written with SPECTRAL_WINDOW_ID = 0 and the frequencies are read back from window 0, so an MS or caltable describing more than one spectral window is rejected instead of having one window’s gains labelled with another’s channels.

write_caltable duplicates its single solution across the polarisation axis, so collapsing that axis on the way back in is a no-op for TABASCAL’s own tables. A caltable from CASA can hold a genuinely different Jones term per polarisation, and averaging those would return a gain that calibrates neither — so read_caltable() requires the unflagged polarisations to agree and raises otherwise. Per-polarisation reading is tracked by issue #151. A flagged polarisation is treated as missing rather than as zero: where one polarisation holds a solution and the other does not, the surviving one is returned.

What a failed write leaves behind

overwrite=True removes a calibration that took a run to produce, so write_caltable checks every argument before it touches anything on disk — including the checks that would otherwise only fail deep in the write, such as a non-numeric gains array reaching np.isfinite, and overwrite itself, which is required to be a genuine boolean rather than taken on its truthiness: overwrite="False" reads as a refusal and would delete the very table the caller was trying to protect. The gains are also cross-checked against the MS they claim to belong to: the caltable carries a copy of the MS’s ANTENNA and SPECTRAL_WINDOW, and its own rows index those copies, so gains of the wrong antenna or channel width would produce a table that disagrees with the MS inside itself. A mismatch names both counts.

The output path is also required not to overlap the MS. Writing the caltable to the MS, or to a directory containing it, would delete the observation before its subtables could be copied out — and writing it inside the MS means writing into the very directories being copied from. All three are rejected up front.

The check asks the filesystem rather than comparing paths as text, because one directory has many spellings: a symlink, a .., and — on a case-insensitive filesystem such as APFS or NTFS — a different case. realpath hands back whichever spelling it was given, so X.ms and x.ms resolve to strings that differ while naming one directory; identity and containment are settled by (st_dev, st_ino) instead, walking a path’s ancestors rather than testing it as a prefix. A sibling named x.ms2 is therefore not mistaken for a child of x.ms, and a case-variant alias cannot spell its way past the guard.

That gives two guarantees, which are deliberately different:

A caller’s mistake costs nothing — the call raises before the removal, and an existing table is left exactly as it was.

An I/O failure part-way through the write cannot put the old table back. The partial output is then removed on a best-effort basis before the error is re-raised, so a half-written table can only survive a failure that also prevents its own removal. The original error always propagates — nothing raised while clearing up replaces it.

Warning

Do not rely on applycal to set the weights for a frequency-dependent gain. applycal(calwt=True) applies a single per-row weight factor, constant across channels, even when WEIGHT_SPECTRUM exists — it collapses the frequency axis rather than scaling each channel by its own |g_ch|**2. TABASCAL therefore computes WEIGHT_SPECTRUM itself when it writes results.

Reading Measurement Sets, and the calibration tables that go beside them.

Everything that knows the MS format lives here. Named for the format rather than generically (io.py) so that a second input format becomes a sibling module with its own name, instead of accreting into one file the way MS reading accreted into tab_tools.py.

The CASA caltable block at the end is the one place that knows the calibration table format, so that gains can be exchanged with standard tooling (applycal, CASA/CARAcal/stimela) instead of ad-hoc .npz files. It sits here because a caltable is an MS-shaped thing that only means anything beside its MS, and is kept a self-contained block so it can become its own module unchanged.

Its scope is the scope tabascal solves for: one spectral window and one correlation. Both are checked rather than assumed – a multi-window table would label one window’s gains with another’s channels, and a table holding a different Jones term per polarisation cannot be collapsed to the single gain tabascal fits.

Gain convention (CASA’s, and the only one used here)

The gain multiplies the model to give the observed visibility:

V_obs[p, q] = g_p * conj(g_q) * V_true[p, q]

so calibrating divides it out, and the noise follows the data:

V_cal      = V_obs / (g_p conj(g_q))
sigma_cal  = sigma / |g_p conj(g_q)|
weight_cal = weight * |g_p conj(g_q)|**2      (weight == 1 / sigma**2)

A scalar flux scale V_cal = k * V_obs is therefore the antenna-independent gain g = k ** -0.5.

tabascal.ms.CORR_TYPES = {'i': 1, 'll': 8, 'lr': 7, 'q': 2, 'rl': 6, 'rr': 5, 'u': 3, 'v': 4, 'xx': 9, 'xy': 10, 'yx': 11, 'yy': 12}

CASA Stokes enumeration (casacore Stokes.h), for the correlations that can be selected by name. The MS records which of these it holds in POLARIZATION::CORR_TYPE, so a correlation is identified by its code rather than by where it sits on the data axis.

tabascal.ms.DEFAULT_TIME_SCALE = 'utc'

Time scale assumed when an MS does not say which one its TIME column uses.

class tabascal.ms.MSLayout(n_time: int, n_bl: int, a1: ndarray, a2: ndarray)

How an MS partition’s rows map onto tabascal’s (n_time, n_bl) grid.

a1/a2 are the antenna pairs of one timestep’s block, in row order.

a1: ndarray

Alias for field number 2

a2: ndarray

Alias for field number 3

n_bl: int

Alias for field number 1

n_time: int

Alias for field number 0

tabascal.ms.TIME_UNITS = {'d': 'd', 'day': 'd', 'days': 'd', 's': 's', 'sec': 's', 'second': 's', 'seconds': 's', 'secs': 's'}

Units a TIME column can declare in QuantumUnits, mapped to the two tabascal distinguishes. casacore writes 's'; the longer spellings are accepted for the same reason the scale names are – the point is to read whatever the MS declares, not to insist on one spelling of it.

tabascal.ms.apply_gains_to_data(vis: NDArray, gains: NDArray, a1: NDArray, a2: NDArray, sigma: NDArray | float | None = None)

Divide the gains out of the data (and carry the noise with it).

This is the whole convention in one place – see the module docstring.

vis is (n_bl, n_freq, n_time); gains is (n_ant, n_freq, n_time); sigma is anything broadcastable against vis (a scalar, or (n_bl, 1, 1)).

Returns (vis_cal, sigma_cal); sigma_cal is None if no sigma was given. A baseline whose gain is dead – flagged, zero or non-finite – has no calibrated value, and comes back as NaN in both; such visibilities must be flagged by the caller.

tabascal.ms.fitted_correlation(ms_path: str, zarr_corr, corr, n_corr: int, pol_id: int = 0) → int

Index on the MS’s correlation axis that the results belong to.

tabascal fits one correlation. Its name comes from the corr argument if given, else from the corr attribute the run recorded on the results zarr, and is resolved to an index by identity, not by position – a single-polarisation MS holds one correlation whatever it is, so yy is index 0 there.

pol_id is the POLARIZATION row the data partition actually uses, the same one read_ms resolved through DATA_DESCRIPTION. Row 0 is only a convention: a partition on another row may order its correlations differently, or hold fewer of them, and resolving against the wrong row would put the results in the wrong polarisation without a word.

A zarr written before that attribute existed carries no name. With one correlation there is only one answer; with more, guessing would silently write the results into the wrong polarisation, so it is an error.

tabascal.ms.grid_to_rows(arr, n_freq: int, n_corr: int = 1)

tabascal’s (bl, freq, time) back to MS (row, chan, corr).

The exact inverse of rows_to_grid(), and the reason they live next to each other: a transpose written once in each direction cannot drift out of step the way two independent ones can.

tabascal.ms.infer_time_unit(times) → str

"s" or "d" for a TIME column that declares no unit.

Split out from times_to_mjd() so that a caller which wants to reduce one array and convert another can. orbit_config._integration_times_mjd is the one that does: the unit has to be read from every row, because the reduction weighs the times by how often they occur, while the values it returns are the distinct ones. Converting first and deduplicating after would do both, but it scales the whole column to do it – 2.3x its size against 1.3x, on a main table that can run to gigabytes – and division is not injective in binary64, so two raw timestamps can land on one MJD and leave the mean epoch weighted differently.

Works in place on its own copy of the finite times, which no caller shares.

tabascal.ms.into_corr(col, corr_idx: int, n_corr: int, fill)

Place a one-correlation result on the MS’s correlation axis.

Results are (row, chan, 1) while the MS column may be (row, chan, 4). The fitted correlation takes the result; the others take fill – zero for the model columns, and the data column itself for the data-frame columns, which is what “no gain applied and nothing subtracted” means there.

Works on the raw arrays because xarray will not broadcast a length-1 corr dimension against a length-4 one; the caller re-wraps.

tabascal.ms.ms_layout(xds) → MSLayout

Derive and validate the row layout of one MS partition.

tabascal reads every visibility column as (n_time, n_bl), so the rows have to be time-major: n_bl consecutive rows holding one timestep of a fixed baseline sequence, repeated per timestep. That was assumed everywhere the reshape appears; it is checked once here instead, for the reader and the results writer alike.

Three distinct ways an MS can break the reshape, each silent on its own: a baseline-major store repeats one pair down the first rows; a per-timestep reshuffle keeps the row count right while moving each baseline’s data; and rows that cycle through baselines and times together satisfy both of those while landing every visibility on the wrong timestamp.

Nothing the size of a column is ever held in memory. The only values read whole are the n_time distinct times and the first block’s n_bl antenna pairs; the checks over the full columns are reductions, computed together in one pass, chunk by chunk, on dask-backed input.

tabascal.ms.ms_row_chunks(row_chunk=None)

Leave dask-ms defaults intact unless a positive row chunk is requested.

tabascal.ms.partition_noise(xds, n_time: int, n_bl: int, n_freq: int, corr_idx: int = 0, chans=None, n_chan_ms: int | None = None) → ndarray | None

The noise on one MS partition’s visibilities, as resolved as the MS allows.

Most specific column first:

SIGMA_SPECTRUM (row, chan, corr)

Noise per (baseline, channel), shape (n_bl, n_freq). The default, because a bandpass is not flat and an MS that has measured that says so.

SIGMA (row, corr)

Per-baseline noise, shape (n_bl,) – the band-averaged version of the same measurement, and what most MSs carry.

Either column keeps its time axis if it has one to keep: a column whose rows genuinely change over the observation comes back as (n_bl, n_freq, n_time), or (n_bl, 1, n_time) from SIGMA. See tabascal.noise.

A SIGMA_SPECTRUM that is absent, that holds no positive finite value anywhere (a column that was never filled in), or that describes a different set of channels from the ones being read is not an error: the read falls through to SIGMA. A column that contradicts the row layout is an error – that is the reader and the MS disagreeing about the grid, which reading another column instead would only bury.

Nothing is invented: if neither column is usable this returns None, after saying why, because a made-up noise scale silently re-weights the entire fit. None rather than an exception because data.noise is read after the MS – an override is exactly the answer to an MS with no noise in it, and raising here would take its turn away. TabConfig.set_noise() is where a still-unset noise becomes the error that stops the run.

chans narrows SIGMA_SPECTRUM to the channels being read, and must be the same selection the data went through: the noise divides those visibilities cell by cell, so a noise left on the full band would weight every channel by another channel’s. SIGMA has no channel axis to narrow. n_chan_ms is the MS’s own channel count, which the column is validated against; it defaults to n_freq, i.e. no selection was made.

tabascal.ms.partition_polarization(ms_path: str, xds) → int

The POLARIZATION row the partition xds uses.

The half of partition_setup() a caller needs when it is placing correlations rather than reading channel frequencies.

tabascal.ms.partition_setup(ms_path: str, xds) → tuple

(spectral_window_id, polarization_id) for the partition xds.

xds_from_ms partitions by (FIELD_ID, DATA_DESC_ID) and records the id in each partition’s attrs, so a partition can say which subtable rows its data is described by rather than assuming row 0.

tabascal.ms.read_caltable(path: str) → dict

Read a caltable written by write_caltable() (or by CASA).

Returns a dict with gains (n_ant, n_freq, n_time) complex – flagged solutions set to NaN – plus times, ant_idx, freqs (None if the table carries no SPECTRAL_WINDOW) and viscal.

time_ref is the scale the TIME column declares, as written, or None for a table that declares none. keywords holds the caller’s keywords – whatever the solver recorded, such as tabascal’s FittedCorr – with the names the format claims for itself left out, so that reading back what was written needs no casacore.

Reads the single-correlation, single-spectral-window tables tabascal fits. A table whose polarisations carry genuinely different solutions, or which describes more than one spectral window, is an error rather than a silent collapse – see _collapse_pols() and _single_spw_chan_freq().

tabascal.ms.read_time_scale(column_keywords: dict, column: str = 'TIME') → str

Time scale declared by an MS column, from its MEASINFO record.

A Measurement Set records the scale its times are on rather than leaving it to convention: the TIME column carries MEASINFO {'type': 'epoch', 'Ref': 'UTC'}. UTC is overwhelmingly the common case, but it is a declaration to be read, not a property to be assumed – an MS may legitimately declare TAI or another scale, and the difference is the accumulated leap seconds, 37 s since 2017, which is ~285 km along a LEO satellite’s ground track.

Read by read_ms(), which normalises the times it returns onto UTC, and by orbit_config’s preflight epoch helper, which normalises the same way.

Parameters:

• column_keywords (dict) – Per-column keyword mapping, as returned by xds_from_ms(path, column_keywords=True)[1].

• column (str, optional) – Column to read the scale from. Defaults to "TIME".

Returns:

The declared scale, lower-cased, or DEFAULT_TIME_SCALE when the MS does not declare one.

Return type:

str

Warns:

UserWarning – If the column declares no scale and the default is assumed. An assumed scale is worth saying out loud, and worth being able to filter and assert on, which a bare print is not – as in read_time_unit().

tabascal.ms.read_time_unit(column_keywords: dict, column: str = 'TIME') → str | None

Unit an MS column declares its times in, from its QuantumUnits keyword.

The MS format leaves the unit of TIME to the column: casacore writes seconds and declares QuantumUnits ['s'], but days are equally legal. Reading the declaration makes it authoritative and leaves times_to_mjd()’s heuristic as the fallback for the columns that carry no declaration.

Parameters:

• column_keywords (dict) – Per-column keyword mapping, as returned by xds_from_ms(path, column_keywords=True)[1].

• column (str, optional) – Column to read the unit from. Defaults to "TIME".

Returns:

"s" or "d", or None when the MS declares nothing usable – which is not an error, only an absence for the caller to infer around.

Return type:

str or None

Warns:

UserWarning – If the column declares a unit that is neither seconds nor days. An ignored declaration is worth saying out loud, and worth being able to filter and assert on, which a bare print is not.

tabascal.ms.remove_caltable(path: str, ms_path: str) → bool

Delete the calibration table at path, if that is what is there.

For a solution that has been superseded. A run that fits no gains has no table to overwrite the previous run’s with, and one left standing beside the new results reads as the current calibration – the failure mode a stale file always has, and a calibration is a bad thing to be wrong about.

Returns whether a table was removed; nothing there is not an error.

This is a rmtree on a path a caller named, so it is the one call here that has to be sure. Both checks refuse rather than delete: the path must hold a casacore table declaring itself a calibration (_is_caltable()), and it must not be, contain, or sit inside the MS – the same guard write_caltable() applies before it removes an existing output, and for the same reason.

tabascal.ms.resolve_correlation(ms_path: str, corr: str, pol_id: int = 0) → int

Index of corr on the MS’s correlation axis.

Resolved by identity, not by position: the requested correlation is mapped to its CASA Stokes code and located in POLARIZATION::CORR_TYPE.

A full 4-correlation MS lays its correlations out in the conventional order, so a fixed {xx: 0, xy: 1, yx: 2, yy: 3} table happens to work there. It does not generalise: an MS written with a single polarisation holds only that one, so its correlation axis has length 1 whatever the polarisation is, and yy means index 0 rather than 3. A 2-correlation (XX, YY) MS breaks the same table in a different way. Reading CORR_TYPE covers all three, and turns a request for an absent correlation into an error rather than either an index error or a silent read of the wrong polarisation.

Parameters:

• ms_path (str) – Path to the Measurement Set.

• corr (str) – Correlation name, e.g. "xx". Case-insensitive.

• pol_id (int, optional) – Row of POLARIZATION describing the data being read, from resolve_data_description(). Defaults to 0.

Returns:

Position of corr on the data’s correlation axis.

Return type:

int

Raises:

ValueError – If corr is not a recognised name, or the MS does not contain it.

tabascal.ms.resolve_data_description(ms_path: str, data_desc_id: int = 0)

(spectral_window_id, polarization_id) for a DATA_DESC_ID.

An MS does not tie its data to row 0 of SPECTRAL_WINDOW and POLARIZATION. It carries a DATA_DESC_ID per row, and the DATA_DESCRIPTION subtable maps that to the spectral window and polarization setups the data actually uses. Those ids are 0 in the common single-setup case, which is why assuming 0 usually works – and why an MS with several setups would silently read another one’s channel frequencies or correlation layout.

xds_from_ms partitions by (FIELD_ID, DATA_DESC_ID) and records the id in each partition’s attrs, so the caller can say which partition it is reading.

Falls back to (0, 0) with a warning if DATA_DESCRIPTION cannot be read, which keeps a malformed store loadable.

tabascal.ms.rows_to_grid(col, n_time: int, n_bl: int, n_freq: int)

MS (row, chan) to tabascal’s (bl, freq, time).

The rows are time-major – n_bl consecutive rows per timestep, in a fixed baseline order, as ms_layout() checks – so the row axis unfolds into (n_time, n_bl) and the time axis then moves to the back.

Method calls rather than np./jnp./da. functions, so the reader can pass jax arrays and the writer dask ones through the same mapping.

tabascal.ms.times_to_mjd(times, unit: str | None = None) → ndarray

An MS TIME column as Modified Julian Dates in days.

tabascal works in MJD days throughout, while an MS stores TIME in the unit its column declares – seconds, as casacore writes it. Pass that declaration (read_time_unit()) and it is honoured; pass None and the unit is inferred, because not every writer fills the keyword in.

The inference reads the magnitude of the times, and nothing else. An MJD day number is at most ~1e5 in any plausible observing era – 1e5 is the year 2132 and -1e5 is 1585 – while the same instant in seconds is ~1e9. No MS can plausibly sit between the two, so one comparison settles it, for a single integration and a full observation alike. The test is strict: a magnitude of exactly 1e5 reads as days.

The value compared is the median of the finite magnitudes, which is a statement about the column rather than about any one of its entries. A single row can be neither: casacore leaves TIME at zero in a row that was added and never filled, and one such row in a column of seconds drags the smallest magnitude to zero – read as days, the observation then overflows the preflight epoch check. The largest magnitude is no better: a stray value in the other unit, a seconds timestamp left in a column of days, reaches it just as easily. Half the column has to be wrong before a median is, and a median is indifferent to the order its values arrive in, which no caller here guarantees.

Non-finite entries are dropped rather than ranked, so an inf cannot decide the unit from either end. A column with no finite time at all reads as days, so it goes downstream exactly as it arrived – the same nothing, not a nothing scaled by 86400.

The reduction is over the times as given, duplicates included, so that frequency counts – which is what “half the column” means, and what makes a handful of corrupt rows survivable. The alternative, reducing over the distinct values, buys nothing here and costs that: two sentinel rows at 0 and 1 beside any number of real ones are three distinct values whose median is 1, so a column of seconds would read as days.

That leaves the callers to agree by construction rather than by construction of the statistic. Within one partition they do, for every MS ms_layout() accepts: it derives n_time as the number of distinct times and requires the rows to be that many blocks each holding one constant time, so two blocks cannot share one and the TIME.reshape(n_time, n_bl)[:, 0] that read_ms() and write._observation_grid pass holds no duplicates for orbit_config._integration_times_mjd’s np.unique to remove. A column whose blocks repeat a time can part them, but its reshape has already stopped landing on block boundaries, so the unit is the least of it.

The scopes still differ: the preflight helper reads the whole main table through casacore while read_ms() takes one xds_from_ms partition, so a multi-field or multi-SPW MS reaches them as different columns entirely. Weighing by frequency is what makes that survivable – the rows of another field are a minority of a column, not half of it. The values come back in the order they were given: only the decision reduces them.

The spacing of consecutive samples used to decide the unit instead, until issue #208; docs/api/ms.rst records why it could not. In short, an integration of exactly 0.5 s read as days, and so did anything shorter.

One rule for every caller, declared or inferred. All three pass what read_time_unit() gave them – read_ms(), write._observation_grid and the preflight epoch check – so a declaration settles the unit for all of them and this reduction is what is left for an MS that declares nothing. Every caller honouring the declaration is what keeps an MS whose QuantumUnits contradicts its magnitudes from reading one way to the run and another to the TLE age checks.

tabascal.ms.write_caltable(path: str, gains: NDArray, times: NDArray, ms_path: str, interval: float = 0.0, n_pol: int = 2, viscal: str = 'B Jones', overwrite: bool = True, keywords: Mapping | None = None, time_ref: str = 'UTC') → str

Write an applycal-compatible calibration table.

The layout mirrors exactly what casatasks.gaincal emits (verified against a reference table it produced): one row per (time, antenna), time-major, with CPARAM of shape (n_chan, n_pol). B Jones rather than G Jones because the gains here are frequency dependent, which a scalar G table cannot represent.

Verified against CASA: applycal accepts the table, and its CORRECTED_DATA reproduces V / (g_p conj(g_q)) to 6e-7 relative (float32 round-off) over 6.7e6 visibilities.

Do not rely on applycal to set the weights for a frequency-dependent gain. applycal(calwt=True) was measured to apply a single per-row weight factor, constant across channels (within-row CV of the applied factor = 0.0000), even when WEIGHT_SPECTRUM exists – it collapses the frequency axis rather than scaling each channel by its own |g_ch|**2. For a channel-constant gain that is exact; for a frequency-dependent one it is an approximation. tabascal therefore computes WEIGHT_SPECTRUM = 1 / sigma_cal**2 per channel itself when it writes results.

A gain that is zero or non-finite carries no solution, and both halves of that are written: FLAG is set and CPARAM is NaN. A reader going by the flag and one going by the value have to reach the same conclusion – a zero left in CPARAM reads as a solution that calibrates to infinity, and an Inf reads as a number too.

One spectral window only. Every row is written with SPECTRAL_WINDOW_ID = 0, so an MS with more than one window is rejected rather than having one window’s gains filed under another’s id.

Parameters:

• path (str) – Output caltable path.

• gains (NDArray) – (n_ant, n_freq, n_time) complex – g_p, in the V_obs = g_p conj(g_q) V_true convention of the module docstring.

• times (NDArray) – (n_time,) MS TIME values in seconds (MJD seconds, as in the MS), on the scale time_ref names.

• ms_path (str) – The MS these gains belong to; its ANTENNA, FIELD, SPECTRAL_WINDOW, OBSERVATION and HISTORY subtables are copied into the caltable, as CASA does. Subtables it does not have are skipped.

• n_pol (int, optional) – CASA writes 2 polarisations even for a single-correlation MS, so the gain is duplicated across the pol axis by default.

• keywords (dict, optional) – Extra table keywords, written beside the table’s own. For what the solver knows and the format has no field for – the correlation tabascal fitted, say – so that the table still says it when it is read back somewhere else. The names the table needs for itself (_RESERVED_KEYWORDS) are refused rather than overwritten, and so are values casacore cannot be relied on to encode: a string, bool, int or float, or a non-empty list of one of those.

• time_ref (str, optional) – Epoch reference the TIME column declares, which must be the MS’s own: times is a copy of its column and nothing here shifts it, so declaring UTC over a TAI-declared MS moves every timestamp by the accumulated leap seconds for anything that reads the declaration. Validated against TIME_SCALES – the scales tabascal can interpret – and written in casacore’s upper case.

Returns:

• str – The caltable path.

• The gains are checked against the MS they claim to belong to before any of

• this happens, since the copied subtables are what the table’s own rows index

• ANTENNA1 into ANTENNA, and CPARAM’s channel axis onto

• CHAN_FREQ. Gains of the wrong width would produce a table that disagrees

• with the copy of the MS inside itself.

• ``path`` may not be, contain, or sit inside ``ms_path`` (the output is)

• removed before the subtables are copied out of the MS, so an overlapping

• path would destroy the observation. That is rejected before any of it

• happens, which is also what keeps the clean-up on the failure path from

• reaching anything but the caltable’s own directory.

• **``overwrite=True`` replaces a calibration table and nothing else.** An

• existing output that is not one – a caller’s directory, an ordinary file, a

• Measurement Set, a table too damaged for casacore to open (see

• _is_caltable()) – is a ValueError naming the path and what was

• found, rather than a rmtree. Every legitimate overwrite target is a

• previous solution, so anything else there is a path pointing somewhere the

• caller did not mean; the cost of that reading is a deletion that cannot be

• undone. A damaged caltable is refused on the same rule and has to be removed

• by hand.

Raises:

• ValueError – If the arguments do not describe one single-spectral-window solution set for this MS. Every check on the caller’s arguments runs before an existing table is removed: overwrite=True deletes a calibration that took a run to produce, and a caller’s mistake must not cost them that. An I/O failure part-way through the write cannot put the old table back; the partial output is then removed on a best-effort basis before the error is re-raised, so a half-written table can only survive a failure that also prevents its removal. The original exception always propagates – nothing raised during the clean-up replaces it. Also if path exists, overwrite is true, and what is there is not a calibration table this function could have written.

• FileExistsError – If path exists and overwrite is false.