NEW: PayID registration, PayTo payment initiation and Account inquiries issues - [incident first identified 21 November /1:25PM in AEST/AEDT timezone])

Incident Report for Azupay

Resolved

We believe the issue is now resolved.

The underlying root cause was extended response times for requests our systems sent to our upstream NPP banking provider.

If you are still encountering issues, please do reach out using our service desk: https://azupay.atlassian.net/servicedesk/customer/portal/3 including any pertinent information such as PayID's, bsb/account numbers, date/time stamps and amounts.

Please do subscribe to our status page for the latest and most accurate information.

We apologise for the inconvenience caused.

Posted Nov 22, 2024 - 03:11 AEDT

Monitoring

We believe the issue is now resolved and will continue closely monitoring the performance of the system.

If you are still encountering issues, please do reach out using our service desk: https://azupay.atlassian.net/servicedesk/customer/portal/3 including any pertinent information such as PayID's, bsb/account numbers, date/time stamps and amounts.

Please do subscribe to our status page for the latest and most accurate information.

We apologise for the inconvenience caused.

Posted Nov 22, 2024 - 02:13 AEDT

Update

Our analysis shows the following impacts:

This isn't a cyber incident.

Elevated number of errors:
* BSB enquiry
* PayID enquiry

Delays in processing:
* Creation/Update of PayID's via api
* PayTo payment agreement creation
* PayTo payment initiations

Our recommendations are as follows:

* Please inform your customers of delays where applicable.

Please do subscribe to our status page for the latest and most accurate information.

We apologise for the inconvenience caused.

Posted Nov 22, 2024 - 01:41 AEDT

Update

Posted Nov 22, 2024 - 01:09 AEDT

Update

Posted Nov 22, 2024 - 00:39 AEDT

Update

Posted Nov 22, 2024 - 00:07 AEDT

Update

Posted Nov 21, 2024 - 23:39 AEDT

Update

Posted Nov 21, 2024 - 23:03 AEDT

Update

Posted Nov 21, 2024 - 22:31 AEDT

Update

Posted Nov 21, 2024 - 21:55 AEDT

Update

Our analysis shows the following impacts:

This isn't a cyber incident.

Elevated number of errors:
* BSB enquiry
* PayID enquiry

Delays in processing:
* Creation of PayID's via api
* PayTo payment agreement creation
* PayTo payment initiations

Our recommendations are as follows:

* Please inform your customers of delays where applicable.

Please do subscribe to our status page for the latest and most accurate information.

We apologise for the inconvenience caused.

Posted Nov 21, 2024 - 21:28 AEDT

Update

Posted Nov 21, 2024 - 21:01 AEDT

Update

We continue to encounter issues with our NPP banking provider and we are working with them to resolve it.

The following services as affected:

• PayID Resolution
• Account Resolution
• PayID Enquiry
• Register PayID
• Update PayID Name
• Update PayID Status

As Azupay registers PayIDs asynchronously for payment requests, for in flight payment requests raised after the incident begun at 19:25 AEST customers using our payment request app may see a payID checkout but will be unable to process these payment requests with their bank, as the PayIDs will not yet be registered on the NPP network.

These requests are being queued by our system and will be registered once our NPP banking partner recovers relevant services.

Payment Requests raised prior to the incident should operate as expected and should be able to receive payments.

Posted Nov 21, 2024 - 20:27 AEDT

Identified

We have had a recurrence of this issue with our NPP banking provider again and we are working with them to resolve it.

The following services as affected::

• PayID Resolution
• Account Resolution
• PayID Enquiry
• Register PayID
• Update PayID Name
• Update PayID Status

As Azupay registers PayIDs asynchronously for payment requests, for in flight payment requests raised after the incident begun at 19:25 AEST customers using our payment request app may see a payID checkout but will be unable to process these payment requests with their bank, as the PayIDs will not yet be registered on the NPP network.

These requests are being queued by our system and will be registered once our NPP banking partner recovers relevant services.

Payment Requests raised prior to the incident should operate as expected and should be able to receive payments.

Posted Nov 21, 2024 - 19:40 AEDT

Monitoring

Monitoring indicates our NPP banking partner services have recovered. We are actively monitoring for any further issues.
Queued Payment Requests have been re-processed, and affected services should now be available for use.

Please raise any ongoing issues encountered via our service desk at https://azupay.atlassian.net/servicedesk/customer/portal/3

We apologise for any inconvenience caused during this outage.

Posted Nov 21, 2024 - 18:37 AEDT

Identified

We have had a recurrence of this issue with our NPP banking provider and we are working with them to resolve it.

The following services as affected::

• PayID Resolution
• Account Resolution
• PayID Enquiry
• Register PayID
• Update PayID Name
• Update PayID Status

As Azupay registers PayIDs asynchronously for payment requests, for in flight payment requests raised after the incident begun at 17:40 AEST customers using our payment request app may see a payID checkout but will be unable to process these payment requests with their bank, as the PayIDs will not yet be registered on the NPP network.

These requests are being queued by our system and will be registered once our NPP banking partner recovers relevant services.

Payment Requests raised prior to the incident should operate as expected and should be able to receive payments.

Posted Nov 21, 2024 - 18:07 AEDT

Monitoring

Monitoring indicates our NPP banking partner services have recovered. We are actively monitoring for any further issues.
Queued Payment Requests are being re-processed, and affected services should now be available for use.

Please raise any ongoing issues encountered via our service desk at https://azupay.atlassian.net/servicedesk/customer/portal/3

We apologise for any inconvenience caused during this outage.

Posted Nov 21, 2024 - 14:45 AEDT

Identified

Our NPP banking partner has identified all affected services for this Incident and have confirmed clients will experience processing issues with the following:

• PayID Resolution
• Account Resolution
• PayID Enquiry
• Register PayID
• Update PayID Name
• Update PayID Status

As Azupay registers PayIDs asynchronously for payment requests, for in flight payment requests raised after the incident begun at 1:25 AEST customers using our payment request app may see a payID checkout but will be unable to process these payment requests with their bank, as the PayIDs will not yet be registered on the NPP network.

These requests are being queued by our system and will be registered once our NPP banking partner recovers relevant services.

Payment Requests raised prior to the incident should operate as expected and should be able to receive payments.

Our NPP banking partner believe they have identified the root cause and are in the process of deploying a break fix. We are seeing recovery of services occurring via relevant monitoring. We will provide updates as services recover and these in flight payment requests are successfully registered.

Posted Nov 21, 2024 - 14:19 AEDT

Update

We are continuing to investigate this issue.

Posted Nov 21, 2024 - 13:33 AEDT

Investigating

We are aware of issues in processing payID registrations, account inquiries and payTo payment initiations. We are currently investigating the impact and will provide a detailed update as soon as one becomes available. This is an upstream issue with our banking partner and we are working with their team for a resolution as soon as possible.

Payment Requests and Payments are not believed to currently be affected.

Please do subscribe to our status page for the latest and most accurate information.

We apologise for the inconvenience caused.

Posted Nov 21, 2024 - 13:32 AEDT

This incident affected: External NPP Network.