Data models

What characterizes an event?

The different types of events in GraceDB are distinguished by the following parameters:

  • Group: the working group responsible for finding the candidate
    • values: CBC, Burst, External, Test
  • Pipeline: the data analysis software tool used make the detection
    • values: MBTAOnline, CWB, CWB2G, gstlal, spiir, HardwareInjection, Fermi, Swift, SNEWS, oLIB
  • Search: the search activity which led to the detection
    • values: AllSky, AllSkyLong, LowMass, HighMass, GRB, Supernova, MDC, BBH

An individual “event stream” is specified by setting the values of these three parameters. For example, choosing Group=CBC, Pipeline=gstlal, and Search=LowMass selects the event stream consisting of low-mass inspiral events detected by the gstlal pipeline from the CBC group. This framework was chosen in order avoid situations where events from different sources would overlap in searches and alerts.

Base event model

In addition to the three parameters described above, there are additional common attributes for all events. These are:

  • submitter: the user who submitted the event
  • created: the time at which the event was created
  • instruments: the interferometers involved in the detection
  • far: the false alarm rate in Hz
  • gpstime: the time at which the event occurred (a.k.a. “Event time”)

The base event class was created with GW events in mind, so not all of the fields will be applicable for any given event. For example, instruments and far do not apply to a Swift GRB event.

Event subclasses

Most events also have pipeline-specific attributes, and these are reflected in event subclasses. For example, the gstlal pipeline produces an estimate for the chirp mass, which is represented in the CoincInspiral event subclass.

Serialized events

Event objects are serialized into JSON format in responses from the API and in LVAlert messages. Here, we show some examples for the different event subclasses.

CBC pipelines (gstlal, spiir, PyCBC, MBTAOnline)

{
    "created": "2018-09-19 18:28:46 UTC",
    "extra_attributes": {
        "CoincInspiral": {
            "combined_far": 0.000368007562149924,
            "end_time": 971609248,
            "end_time_ns": 151740542,
            "false_alarm_rate": 1.0,
            "ifos": "H1,L1,V1",
            "mass": 3.68682909011841,
            "mchirp": 0.0,                                                                                                                                                                                     
            "minimum_duration": 46.848259458,
            "snr": 9.31793628458239
        },
        "SingleInspiral": [
            {
                "chisq": 0.78878409,
                "chisq_dof": 1,
                "coa_phase": 0.97073036,
                "eff_distance": 173.30727,
                "end_time": 971609248,
                "end_time_ns": 151740542,
                "ifo": "H1",
                "mass1": 2.6161611,
                "mass2": 1.070668,
                "sigmasq": 802669.916849115,
                "snr": 5.169538,
                "spin1z": -0.91339749,
                "spin2z": 0.02086957
            },
            {
                "chisq": 1.1967639,
                "chisq_dof": 1,
                "coa_phase": 2.469281,
                "eff_distance": 162.4119,
                "end_time": 971609248,
                "end_time_ns": 153169206,
                "ifo": "L1",
                "mass1": 2.6161611,
                "mass2": 1.070668,
                "sigmasq": 802669.916849115,
                "snr": 5.5163355,
                "spin1z": -0.91339749,
                "spin2z": 0.02086957
            },
            {
                "chisq": 0.851273,
                "chisq_dof": 1,
                "coa_phase": 2.7670317,
                "eff_distance": 164.47913,
                "end_time": 971609248,
                "end_time_ns": 144829104,
                "ifo": "V1",
                "mass1": 2.6161611,
                "mass2": 1.070668,
                "sigmasq": 802669.916849115,
                "snr": 5.4470043,
                "spin1z": -0.91339749,
                "spin2z": 0.02086957
            }
        ]
    },
    "far": 0.000368007562149924,
    "far_is_upper_limit": false,
    "gpstime": 971609248.151741,
    "graceid": "T0132",
    "group": "Test",
    "instruments": "H1,L1,V1",
    "labels": [],
    "likelihood": 9.91315304082986,
    "links": {
        "emobservations": "https://gracedb.ligo.org/api/events/T0132/emobservation/",
        "files": "https://gracedb.ligo.org/api/events/T0132/files/",
        "labels": "https://gracedb.ligo.org/api/events/T0132/labels/",
        "log": "https://gracedb.ligo.org/api/events/T0132/log/",
        "neighbors": "https://gracedb.ligo.org/api/events/T0132/neighbors/",
        "self": "https://gracedb.ligo.org/api/events/T0132",
        "tags": "https://gracedb.ligo.org/api/events/T0132/tag/"
    },
    "nevents": 3,
    "offline": false,
    "pipeline": "gstlal",
    "search": "LowMass",
    "submitter": "albert.einstein@LIGO.ORG",
    "superevent": null
}

CWB

{
    "created": "2018-09-19 18:28:37 UTC",
    "extra_attributes": {
        "MultiBurst": {
            "amplitude": 5.017162,
            "bandwidth": 256.0,
            "central_freq": 1392.169556,
            "confidence": null,
            "duration": 0.023438,
            "false_alarm_rate": null,
            "ifos": "H1,L1",
            "ligo_angle": null,
            "ligo_angle_sig": null,
            "ligo_axis_dec": 69.422546,
            "ligo_axis_ra": 201.224625,
            "peak_time": null,
            "peak_time_ns": null,
            "single_ifo_times": "1042312876.5073,1042312876.5090",
            "snr": 7.28010988928052,
            "start_time": 1042312876,
            "start_time_ns": 500000000
        }    
    },   
    "far": 0.00019265,
    "far_is_upper_limit": false,
    "gpstime": 1042312876.509,
    "graceid": "T0127",
    "group": "Test",
    "instruments": "H1,L1",
    "labels": [],  
    "likelihood": 53.0,
    "links": {
        "emobservations": "https://gracedb.ligo.org/api/events/T0127/emobservation/",
        "files": "https://gracedb.ligo.org/api/events/T0127/files/",
        "labels": "https://gracedb.ligo.org/api/events/T0127/labels/",
        "log": "https://gracedb.ligo.org/api/events/T0127/log/",
        "neighbors": "https://gracedb.ligo.org/api/events/T0127/neighbors/",
        "self": "https://gracedb.ligo.org/api/events/T0127",
        "tags": "https://gracedb.ligo.org/api/events/T0127/tag/"
    },   
    "nevents": null,
    "offline": false,
    "pipeline": "CWB",
    "submitter": "albert.einstein@LIGO.ORG",
    "superevent": null 
}

oLIB

{
    "created": "2018-09-19 18:29:20 UTC", 
    "extra_attributes": {
        "LalInferenceBurst": {
            "bci": 1.111, 
            "bsn": 7.19, 
            "frequency_mean": 721.23, 
            "frequency_median": 718.03, 
            "hrss_mean": 8.12e-23, 
            "hrss_median": 2.19e-23, 
            "omicron_snr_H1": 4.98, 
            "omicron_snr_L1": 4.99, 
            "omicron_snr_V1": null, 
            "omicron_snr_network": 6.91, 
            "quality_mean": 15.2, 
            "quality_median": 15.1
        }
    }, 
    "far": 7.22e-06, 
    "far_is_upper_limit": false, 
    "gpstime": 1216336200.66, 
    "graceid": "T0144", 
    "group": "Test", 
    "instruments": "H1,L1", 
    "labels": [], 
    "likelihood": null, 
    "links": {
        "emobservations": "https://gracedb.ligo.org/api/events/T0144/emobservation/", 
        "files": "https://gracedb.ligo.org/api/events/T0144/files/", 
        "labels": "https://gracedb.ligo.org/api/events/T0144/labels/", 
        "log": "https://gracedb.ligo.org/api/events/T0144/log/", 
        "neighbors": "https://gracedb.ligo.org/api/events/T0144/neighbors/", 
        "self": "https://gracedb.ligo.org/api/events/T0144", 
        "tags": "https://gracedb.ligo.org/api/events/T0144/tag/"
    }, 
    "nevents": 1, 
    "offline": false, 
    "pipeline": "oLIB", 
    "submitter": "albert.einstein@LIGO.ORG", 
    "superevent": null
}

External

{
    "graceid": "E8840",
    "gpstime": 1123421619.0,
    "pipeline": "SNEWS",
    "group": "External",
    "extra_attributes": {
        "GRB": {
            "redshift": null,
            "coord_system": "UTC-FK5-GEO",
            "trigger_id": "1012984",
            "trigger_duration": null,
            "observatory_location_id": "GEOLUN",
            "author_ivorn": "ivo://nasa.gsfc.tan/gcn",
            "author_shortname": "SNEWS (via VO-TAN)",
            "how_reference_url": "http://gcn.gsfc.nasa.gov/snews.html",
            "designation": null,
            "how_description": "SNEWS: SuperNova Early Warning System",
            "error_radius": 360.0,
            "ra": 0.0,
            "dec": 0.0,
            "T90": null,
            "ivorn": "ivo://nasa.gsfc.gcn/SNEWS#Event2018-09-01T14:01:01.00_1000194-580"
        }
    },
    "links": {
        "neighbors": "https://gracedb.ligo.org/api/events/E8840/neighbors/",
        "files": "https://gracedb.ligo.org/api/events/E8840/files/",
        "log": "https://gracedb.ligo.org/api/events/E8840/log/",
        "tags": "https://gracedb.ligo.org/api/events/E8840/tag/",
        "self": "https://gracedb.ligo.org/api/events/E8840",
        "labels": "https://gracedb.ligo.org/api/events/E8840/labels/",
        "emobservations": "https://gracedb.ligo.org/api/events/E8840/emobservation/"
    },
    "created": "2018-09-01 16:00:12 UTC",
    "far": null,
    "instruments": "",
    "labels": [],
    "search": "GRB",
    "nevents": null,
    "superevent": null,
    "submitter": "albert.einstein@ligo.org",
    "offline": false,
    "likelihood": null,
    "far_is_upper_limit": false
}

Superevents

What is a superevent?

In reality, what we called “events” above might be better characterized as “triggers”, since different analysis pipelines may detect the same physical GW event and submit it to GraceDB. In order to collect the information about a single physical event in one place, reduce the amount of follow-up processing needed, and issue alerts about only one GraceDB entry per physical event, we have created the “superevent” data model.

A downstream process which is separate from GraceDB listens for event creations, analyzes their attributes, and determines how to aggregate events into superevents.

Data model

The main attributes of the superevent data model are:

  • superevent_id: a unique date-based ID (Example: S180912b; more information below in Date-based IDs)
  • gw_id: a unique date-based ID only assigned to superevents which are confirmed GWs (Example: GW180915BC; more information below in Date-based IDs)
  • category: superevent category (Production, Test, or MDC); more information below in Categories
  • gw_events: list of graceids corresponding to Event objects which are part of this superevent and were submitted by GW analysis pipelines
  • em_events: list of graceids corresponding to Event objects which are part of this superevent and are in the “External” group (i.e., were observed by electromagnetic or neutrino telescopes)
  • created: time at which the superevent was created
  • submitter: user who created the superevent
  • t_start: time corresponding to start of window for grouping events into this superevent
  • t_end: time corresponding to end of window for grouping events into this superevent
  • t_0: best estimate of time at which the GW event occurred

Serialized superevent

Here is an example of a superevent which has been serialized into a JSON:

{
    "category": "Test",
    "created": "2018-09-19 18:28:54 UTC",
    "em_events": [
        "E0155"
    ],
    "far": 0.0000342353,
    "gw_events": [
        "T0133",
        "T0142"
    ],
    "gw_id": null,
    "labels": [],
    "links": {
        "emobservations": "https://gracedb.ligo.org/api/superevents/TS170101z/emobservations/",
        "events": "https://gracedb.ligo.org/api/superevents/TS170101z/events/",
        "files": "https://gracedb.ligo.org/api/superevents/TS170101z/files/",
        "labels": "https://gracedb.ligo.org/api/superevents/TS170101z/labels/",
        "logs": "https://gracedb.ligo.org/api/superevents/TS170101z/logs/",
        "self": "https://gracedb.ligo.org/api/superevents/TS170101z/",
        "voevents": "https://gracedb.ligo.org/api/superevents/TS170101z/voevents/"
    }, 
    "preferred_event": "T0133",
    "submitter": "albert.einstein@LIGO.ORG",
    "superevent_id": "TS170101z",
    "t_0": 1167264018.0,
    "t_end": 1167264025.0,
    "t_start": 1167264014.0
}

Categories

There are three categories of superevents:

  • Production: “real” superevents which correspond to potential GW events
  • Test: used for generic testing superevent creation, annotation, etc.
  • MDC: generated as part of the mock data challenge (MDC), which comprises a constant stream of events which are sent to GraceDB for testing by internal and LV-EM users

Each category of superevent may only contain events of the corresponding type; i.e., Production superevents may only contain production (G, E, and H-type) events, Test superevents may only contain test (T-type) events, and MDC superevents may only contain MDC (M-type) events.

Letter suffixes for superevent date-based IDs are also calculated independently for each superevent category (see Date-based IDs).

Date-based IDs

We generate IDs for superevents based on the date of their occurrence, at the time of creation. These IDs have three parts:

  • Prefix
  • Six digit (YYMMDD) date string
  • Letter suffix

A few examples are:

  • S180920abc
  • TS180717b
  • GW170817A

Prefix

The prefix is determined by the superevent’s “status” (is it marked as a “confirmed GW” or not) and its category. A superevent’s category will never change, but its status can, resulting in a prefix change. When this happens, the superevent will be accessible (via URLs, the API, searches, etc.) by both its old and new IDs.

Category Not a confirmed GW Confirmed GW
Production S GW
Test TS TGW
MDC MS MGW

Date string

The date string is determined by the superevent’s value for t_0 at creation. This is converted from GPS to UTC time and turned into a YYMMDD string. Note that a superevent’s t_0 value may change as additional events are added to it, but this will not change the date string in the superevent’s ID.

Because these date strings are only 6 digits, they are degenerate over a period of 100 years. We currently define this period to run from 1980-01-01 00:00:00 UTC to 2079-12-31 23:59:59 UTC.

If GraceDB is still around at that point, we will probably have to switch to four-digit years.

Letter suffix

The letter suffix is determined by the chronological ordering of superevents observed on a given date by creation time in GraceDB, not by the occurence time of the actual GW event. This is because a unique ID must be determined at creation time and it is not realistic to recalculate it every time a new superevent is created.

The letter suffix just corresponds to the superevent’s number within this chronological ordering. To be explicit: 1 = a, 2 = b, 27 = aa, and so on.

This suffix is calculated independently for each category of superevent. This means that the first Production superevent for a given date will have a suffix of a, and so will the first Test superevent for that date. In this case, ID uniqueness is preserved by the fact that these categories have different prefixes.

When a superevent is confirmed as a GW, a new letter suffix is determined based on how many superevents for the same date have already been confirmed as a GW. Again, this is dependent on the chronological ordering of the time at which this confirmation action takes place, and not by the occurrence time of the actual GW event. Uppercase letters are used for suffixes for confirmed GWs.

Example scenario

Here’s a hypothetical series of superevent creations occurring on March 31, 2018:

  • A Production superevent is created at 12:00:00 UTC with a t_0 corresponding to 11:30:00 UTC
  • Another Production superevent is created at 12:05:00 with a t_0 corresponding to 11:00:00 UTC
  • A Test superevent is created at 12:10:00 with a t_0 corresponding to 12:05:00 UTC
  • A third Production superevent is created at 12:15:00 with a t_0 corresponding to 10:00:00 UTC

The resulting set of superevent IDs will be:

Creation time (UTC) t_0 (UTC) Category Superevent ID
12:00:00 11:30:00 Production S180331a
12:05:00 11:00:00 Production S180331b
12:10:00 12:05:00 Test TS180331a
12:15:00 10:00:00 Production S180331c

Now let’s say S180331b is confirmed as a GW at 13:00:00 UTC and S180331a is confirmed as a GW at 13:05:00 UTC. This would result in the following superevent IDs:

Creation time (UTC) t_0 (UTC) Category Superevent ID
12:00:00 11:30:00 Production GW180331B
12:05:00 11:00:00 Production GW180331A
12:10:00 12:05:00 Test TS180331a
12:15:00 10:00:00 Production S180331c

Note that the “upgraded” superevents would still be accessible through the web interface, API calls, searches, etc. by either of their IDs.

Annotations

Annotations are pieces of information about an event or superevent that that are added after the event or superevent is created. They are often the results of followup processes, but are sometimes also provided by the same data analysis pipeline that initially generated the event. The most common type of annotation is an log message with the following fields:

  • submitter: the user who created the log message
  • created: the time at which the log message was created
  • filename: the name of the attached file (if applicable)
  • file_version: the specific version of the file for this message
  • comment: the log message text

If the uploaded file is an image, it is displayed along with the comment in the GraceDB event or superevent page. Log messages can also be tagged in order to give other users an idea of the thematic category to which the message belongs. Users can invent arbitrary tags, but the following set have a special status, as they affect the display of information in the event page (i.e., they are “blessed”):

  • analyst_comments: Analyst Comments
  • em_follow: EM Followup
  • psd: Noise Curves
  • data_quality: Data Quality
  • sky_loc: Sky Localization
  • background: Background Information
  • ext_coinc: External Coincidence
  • strain: Strain Data
  • tfplots: Time-Frequency Info
  • pe: Parameter Estimation
  • sig_info: Significance Info
  • audio: Sound Files

Other types of annotations are labels, VOEvent objects, EM observation records, and signoffs.

Serialized annotations

Here we show some examples of annotations which have ben serialized into JSON format. These annotations are basically identical whether they are attached to an event or superevent, and these examples show some of each case.

Log

{
    "N": 1,
    "comment": "Initial data upload",
    "created": "2018-09-19 18:29:01 UTC",
    "file": "https://gracedb.ligo.org/api/events/T0140/files/upload.data.gz,0",
    "file_version": 0,
    "filename": "upload_data.gz",
    "issuer": "albert.einstein@LIGO.ORG",
    "self": "https://gracedb.ligo.org/api/events/T0140/logs/1/",
    "tag_names": ["em_follow", "sky_loc"]
}

Label

{
    "created": "2018-09-19 18:29:24 UTC", 
    "creator": "albert.einstein@LIGO.ORG", 
    "name": "DQV", 
    "self": "https://gracedb.ligo.org/api/events/T0140/labels/DQV"
}

Signoff

{
    "comment": "Everything looks good",
    "instrument": "",
    "self": "https://gracedb.ligo.org/api/superevents/TGW800106H/signoffs/ADV/",
    "signoff_type": "ADV",
    "status": "OK",
    "submitter": "albert.einstein@LIGO.ORG"
}

Notes: - status: can be “OK” or “NO” - signoff_type: can be “ADV” (EM advocate) or “OP” (instrument control room operator) - instrument: two-letter instrument code, like “H1”, “L1”, “V1”, etc. For advocate signoffs, this is an empty string.

EMObservation

{
    "N": 1, 
    "comment": "message", 
    "created": "2018-09-19 18:28:57 UTC", 
    "dec": 6.5, 
    "decWidth": 3.6999999999999993, 
    "footprint_count": 4, 
    "footprints": [
        {
            "N": 4, 
            "dec": 8.0, 
            "decWidth": 0.7, 
            "exposure_time": 1, 
            "ra": 4.0, 
            "raWidth": 0.5, 
            "start_time": "2018-09-19 13:29:00 UTC"
        }, 
        {
            "N": 3, 
            "dec": 7.0, 
            "decWidth": 0.7, 
            "exposure_time": 1, 
            "ra": 3.0, 
            "raWidth": 0.5, 
            "start_time": "2018-09-19 13:28:59 UTC"
        }, 
        {
            "N": 2, 
            "dec": 6.0, 
            "decWidth": 0.7, 
            "exposure_time": 1, 
            "ra": 2.0, 
            "raWidth": 0.5, 
            "start_time": "2018-09-19 13:28:58 UTC"
        }, 
        {
            "N": 1, 
            "dec": 5.0, 
            "decWidth": 0.7, 
            "exposure_time": 1, 
            "ra": 1.0, 
            "raWidth": 0.5, 
            "start_time": "2018-09-19 13:28:57 UTC"
        }
    ], 
    "group": "ARI LJMU", 
    "ra": 2.5, 
    "raWidth": 3.5, 
    "submitter": "albert.einstein@LIGO.ORG"
}

VOEvent

{
    "N": 1,
    "created": "2018-09-19 18:29:08 UTC",
    "file_version": 0,
    "filename": "TGW800106H-1-Preliminary.xml",
    "issuer": "albert.einstein@LIGO.ORG",
    "ivorn": "ivo://gwnet/gcn_sender#TGW800106H-1-Preliminary",
    "links": {
        "file": "https://gracedb.ligo.org/api/superevents/TGW800106H/files/TGW800106H-1-Preliminary.xml,0",
        "self": "https://gracedb.ligo.org/api/superevents/TGW800106H/voevents/1/"
    },
    "voevent_type": "PR"
}