High error rate on nights storage

Timeline

• The table managing our sleepscore overflowed an int32 signed integer foreign key column.
• An ALTER was started but due to the size of the table we changed strategy and added a new column as BIGINT.
• The Index on this table was then added.
• Service was restored once an “ANALYZE” was performed on the table.
• Backfill process was then started to fill the old values of the new column.

Remediation Actions

• Monitoring of near depleted int32 columns already exists on primary keys, but monitoring needs to be added on foreign keys.