PSRCHIVE user documentation: paz

1.0 Purpose

The Pulsar Archive Zapper paz combines a number of manual and automatic modes of interference excision. Lists and/or ranges of frequency channels and/or sub-integrations to be excised can be specified on the command line. Automatic interference excision algorithms are described in section 3 below.

2.0 Usage

For a full list of options, use: 
paz -h
Note that there are three kinds of zapping:
  1. Remove sub-integration data from the archive - an irreversible process; e.g.
    • paz -x ... (extract specified sub-integrations; i.e. delete all sub-integrations except those specified)
    • paz -s ... (delete specified sub-integrations).
  2. Set the weight of sub-integration or frequency channel to zero - a reversible process; e.g.
    • paz -w or -W ... (zero weight specified sub-integrations)
    • paz -z or -Z ... (zero weight specified frequency channels).
  3. Clean corrupted phase bins using some statistical model; e.g.
    • paz -p ... (interpolate over every pth phase bin)
    • paz -L (mow the lawn; see below).

3.0 Algorithms

3.1 Median Smoothed Difference Channel Zapping

Use paz -r to enable an automatic channel zapping algorithm based on tolerance to differences between the bandpass and a median smoothed version of the bandpass.

By default, a median smoothing window 21 channels wide is applied, and all frequency channels with total flux greater than 4 standard deviations away from the median will have their weight set to zero.

Note: standard deviation is defined recursively. That is, the algorithm works as follows

  1. compute median smoothed bandpass
  2. compute standard deviation, ignoring any zapped channels
  3. zap channels greater than tolerated distance from median
  4. if any channel were zapped, goto 2
  5. stop
To change the default parameters, use the command language interpreter, psrsh. For example, the following script sets the median smoothing window width to 13 channels and the tolerance to 6 standard deviations before applying the algorithm and unloading the result with a new extension. 
#!/usr/bin/env psrsh

zap median window=13
zap median cutoff=6
zap median
unload ext=zz

3.2 Profile Lawn Mowing

Use paz -L to enable an algorithm that replaces spikey phase bins with the local median plus noise.

For each sub-integration, the zero-DM total integrated profile is formed and median smoothed with a duty cycle of 0.02 turns. The difference between the profile and its median smoothed self is searched for spikes greater than 4 standard deviations, where the standard deviation is computed using a recursive algorithm that discards both spikes and zeroes (a large number of phase bins tend to equal the local median).

If spikes are found then, for each frequency channel, the off-pulse baseline in the total intensity profile is found and, for each polarization:

  1. the standard deviation of the off-pulse baseline is computed;
  2. the median smoothed profile (0.02 duty cycle) is formed; and
  3. each spike is replaced by the local median plus random noise.
Random noise is generated by selecting a random phase bin and computing the difference between the pulse profile and the median smoothed version of the profile at that random phase bin. If the random phase bin is not also flagged as a spike and the difference is less than 2.5 times the standard deviation of the off-pulse baseline, then the phase bin that has been flagged as a spike will be replaced by its local median plus the difference from the random phase bin.

4.0 Testing and examples

For example, to zero-weight the first three subints: 

paz -w "0 1 2" *.ar
Note: Frequency channel and sub-integration indeces start at zero.

5.0 Known bugs and features that require implementation

  • None at this time.