Friends Project - Datacenter power outage affecting many components. – Incident details

Datacenter power outage affecting many components.

ResolvedMajor outage18 hours 35 minutes

Affected

Main: Game Server

Major outage from 6:00 PM to 12:36 PM

Components

Major outage from 6:00 PM to 12:36 PM

Backbone API & BAC

Major outage from 6:00 PM to 12:36 PM

Main Server: Components

Major outage from 6:00 PM to 12:36 PM

Teamspeak 3

Major outage from 6:00 PM to 12:36 PM

Sentry

Major outage from 6:00 PM to 12:36 PM

  • ResolvedUTC

    We've recovered the TS3 Server.

    -- For this incident, We've losing all the trust to our datacenter. We'll find other datacenter that have suit all the requirement we want (uptime, network, security, etc.)

  • UpdateUTC

    We've got the license and started to recovering the teamspeak server.

  • UpdateUTC

    We've recovered Sentry server.

    We're still waiting new TS3 server license key from our sponsor.

  • UpdateUTC

    We've issued the OS reinstallation for Sentry.

    For the TS3 Server, We're waiting for our sponsor to provide a license key.

  • UpdateUTC

    The server is come back on but still unable to boot into its OS.

    We're continuing trying to recover the server.

  • UpdateUTC

    We still waiting a response from the datacenter regarding one of our server physical status that handle two component.

  • UpdateUTC

    TS3 Server and Sentry seem to don't want to go back on yet. We're contacting our datacenter support to physically check the status of the server.

  • MonitoringUTC

    The datacenter power has come back on. But all servers affected in this incident have to be turned on again.

    We're recovering our service as soon as possible. Sorry for any inconvenience this may cause.

  • UpdateUTC

    Our datacenter is still fixing their UPS (uninterruptible power supply, uninterruptible power source) issues and don't have any update regarding this incident yet.

  • UpdateUTC

    Our datacenter is still fixing their UPS issues and don't have any update regarding this incident yet.

  • UpdateUTC

    Our datacenter is currently having UPS fault and the IDC is fixing their problem.

  • IdentifiedUTC

    Looks like our datacenter or our core router is having an outage and causing these issues.

    We're contacting our datacenter to find the root cause of this incident.

  • UpdateUTC

    We are currently investigating this incident.

  • InvestigatingUTC

    We're contacting our datacenter to find the root cause of this incident.