At Junify, we appreciate that our service is a mission-critical system for many of our customers, and we must strive to keep our service up and running 24/7. Following the principle, we have increased the reliability of our service by introducing redundant infrastructures as much as possible. However, unexpected and uncontrolled incidents can still happen because our service is delivered on the internet, which is composed of many pieces of software. This document describes the classifications of the theoretically possible incidents, and how we detect them once they occur, and how we prepare for them. Our primary goal in this plan is to keep our customers able to continue their operations through potential outages. The document also describes how our customers can get around a disastrous situation in some critical scenarios.
The following two scenarios can cause an event of a complete or partial outage on our platform:
(1) Our upstream service providers, such as AWS, Heroku, or Cloudflare, have an incident, and the impact is propagated to our end users.
(2) Junify adds a new piece of code, and the change causes an outage or degradation
We classify incidents into the following three categories.
| Category | Category Name | Definition |
|---|---|---|
| 1 | Critical | Your business is significantly impacted because Junify’s website is not reachable. A partial outage of our single sign-on (SSO) feature will also be classified in this category. |
| 2 | Major | Your business is moderately impacted because of a partial outage of non-SSO features. |
| 3 | Minor | Non-critical functions are impacted. Bugs on our platform will generally be classified in this category. |
Our website on https://www.junify.com will not be accessible when an upstream service provider becomes unavailable. We use the application monitoring service newrelic to perform health checks on our website every minute. When newrelic detects the website's unavailability, it will notify the engineering team in Junify of the outage via phone call and an email through Pagerduty. In other words, at the latest, we can detect any upstream outages within a minute. Because we receive an incoming call upon a disastrous event, we can respond to incidents at any time. Once an incident occurs, our engineering team will work with the upstream provider for a resolution.
We keep track of any unusual behavior on our platform on Bugsnag. It immediately notifies us of any errors in our service if bugs in our software cause them. If an error is found on our web application, we fix it and release the patch within several hours in most cases. If an error is found on our mobile applications on iOS and/or Android, it usually takes 1–2 business days because they need to go through the review process by Apple and Google. Similarly, it usually takes 1–2 business days to fix an issue on the browser extension.
newrelic, Pagerduty, and Bugsnag described in the previous sections are all independent services. Thus, they still work during events of Junify’s outage.
When an incident of Category-1 or Category-2 occurs, we will communicate with our customers about the incident and potential workarounds until the issue is resolved. The following sections describe them in detail.
We will inform you when a critical or major incident occurs on our platform. Also, we establish an emergency communication channel via phone to talk with Takashi Hamada (CTO) or Hiroaki Yasutake (CEO/CISO) during a critical outage.
When you experience significant issues on our platform, such as being unable to load our website, the first place you should look at is our status page on https://status.junify.com. The status page will be accessible even when Junify’s website becomes unavailable because Atlassian hosts the status page at their service level. It is the main communication channel between you and us where we put the latest information about an incident. We recommend you subscribe to updates when an incident happens on the status page, so you can receive emails if there are any updates.
We will send emails or SMS to all the users when an incident occurs to inform unavailability of our platform. We will also send an update when the issue is resolved and our service is back online again.
You can contact us 24/7 when you experience a critical outage (Category 1). Here is the contact information:
The availability of our platform 24/7 is the most important thing at Junify. However, the system can fail for reasons we cannot control. Therefore, we prepare workarounds for some cases because we understand that some of our features are mission-critical. Although the workarounds are not recommended in normal circumstances, we will help you continue your time-sensitive operations when needed.
The auto-fill feature and SAML login will not be available when an incident of Category 1 occurs. If users can still access their email inboxes outside Junify, they can use the password resetting feature on the web application to log in. In the case of the scenario where SAML authentication is enabled and password authentication is disabled for a web application, we will work with the IT department on your team to use our secondary domain if the incident is in the network layer or temporarily enable password authentication. Upon a request, we could export the list of credentials for each service for each user via email in a way that Junify’s engineering team cannot read the content, just as how our auto-fill feature works on Junify.
We have increased the reliability of our service by employing high-availability components on the upstream services if they are available. They are robust because the same instances are located in multiple regions, and if one becomes unavailable, another instance keeps serving.
We have our standard quality assurance process before releasing new code to the production environment. It consists of thorough automated testing to detect degradation of the software and verification of the behavior and experience by humans.