As all IXOPAY products are now fully functional and stable in the Production Environment, and are pointing to our Primary region, we will close this incident and proceed with updating the Test Environment Incident.
Please keep in mind that you can continue to reach us: •IXOPAY EU Team: Our European support team is available to assist you with any inquiries. You can reach them at +43 1 3530512247. While their focus is on the European market, they'll be happy to route you to the appropriate department if needed. •Email Support: You can also submit your inquiries directly to our email address: support@ixopay.com. We'll respond as soon as possible. •Status Updates:We'll continue to provide regular updates on the situation and any remaining service stabilization efforts at status.tokenex.com
Posted Jul 19, 2024 - 14:26 CDT
Update
As we appreciate your patience and understanding, our teams are continuing efforts to stabilize our Test Environment's degraded performance. We'll continue to provide updates every two hours if any further information becomes available.
Here is how you can continue to reach us: •IXOPAY EU Team: Our European support team is available to assist you with any inquiries. You can reach them at +43 1 3530512247. While their focus is on the European market, they'll be happy to route you to the appropriate department if needed. •Email Support: You can also submit your inquiries directly to our email address: support@ixopay.com. We'll respond as soon as possible. •Status Updates: We'll continue to provide regular updates on the situation and any remaining service stabilization efforts at status.tokenex.com
Posted Jul 19, 2024 - 11:49 CDT
Update
As our teams are continuing efforts to ensure stability within our platform, we are currently experiencing degraded performance within our Test Environment. We'll continue to provide updates if any further information becomes available.
Here is how you can continue to reach us: •IXOPAY EU Team: Our European support team is available to assist you with any inquiries. You can reach them at +43 1 3530512247. While their focus is on the European market, they'll be happy to route you to the appropriate department if needed. •Email Support: You can also submit your inquiries directly to our email address: support@ixopay.com. We'll respond as soon as possible. •Status Updates: We'll continue to provide regular updates on the situation and any remaining service stabilization efforts at status.tokenex.com
Posted Jul 19, 2024 - 10:22 CDT
Update
All IXOPAY products are now fully functional. Our team is actively monitoring performance to ensure continued stability. We'll continue to provide updates if any further information becomes available.
Here is how you can continue to reach us: •IXOPAY EU Team: Our European support team is available to assist you with any inquiries. You can reach them at +43 1 3530512247. While their focus is on the European market, they'll be happy to route you to the appropriate department if needed. •Email Support: You can also submit your inquiries directly to our email address: support@ixopay.com. We'll respond as soon as possible. •Status Updates: We'll continue to provide regular updates on the situation and any remaining service stabilization efforts at status.tokenex.com
Posted Jul 19, 2024 - 09:34 CDT
Update
We understand the inconvenience caused by the recent portal outage. To prioritize resolving the issue as quickly as possible, our US Support Line will be temporarily unavailable.
Our IXOPAY EU Team will be taking over individual customer communications. Please direct all inquiries to support@ixopay.com or +43 1 3530512247. We will continue to provide regular updates at status.tokenex.com.
We are beginning to see products return to a stable state. We will continue to post updates as they become available. Our team is continuing to monitor.
Posted Jul 19, 2024 - 08:37 CDT
Update
We understand the inconvenience caused by the recent outage. To prioritize resolving the issue as quickly as possible, our US Support Line will be temporarily unavailable. Our IXOPAY EU Team will be taking over on individual customer communications.
Please direct all inquires to support@ixopay.com or +43 1 3530512247. We will continue to provide regular updates at status.tokenex.com.
Our teams are continuing efforts but still showing limited availability across all products. We will continue to post updates as they become available.
Posted Jul 19, 2024 - 07:41 CDT
Update
We understand the inconvenience caused by the recent portal outage. To prioritize resolving the issue as quickly as possible, our US Support Line will be temporarily unavailable. Our IXOPAY EU Team will be taking over on individual customer communications.
Please direct all inquires to support@ixopay.com or +43 1 3530512247. We will continue to provide regular updates at status.tokenex.com.
We can confirm we are impacted by the global ongoing incident related to Crowdstrike. Our teams are continuing efforts but still showing limited availability across all products. We will continue to post updates as they become available.
Posted Jul 19, 2024 - 06:49 CDT
Update
We understand the inconvenience caused by the recent portal outage. To prioritize resolving the issue as quickly as possible, our US Support Line will be temporarily unavailable. Our IXOPAY EU Team will be taking over on individual customer communications.
Please direct all inquires to support@ixopay.com or +43 1 3530512247. We will continue to provide regular updates at status.tokenex.com.
Our teams are continuing efforts but still showing limited availability across all products. We will continue to post updates as they become available.
Posted Jul 19, 2024 - 05:58 CDT
Update
Our teams are continuing efforts to stabilize our platform. We will provide updates as products appear stable. We appreciate your patience.
Posted Jul 19, 2024 - 05:20 CDT
Update
Our teams are continuing efforts to stabilize our platform. We will provide updates as products appear stable.
Posted Jul 19, 2024 - 04:46 CDT
Update
We are beginning to see service return to all products, however service may be degraded. Our teams are continuing to ensure our platform is stable.
Posted Jul 19, 2024 - 03:50 CDT
Update
We are seeing service restore for MFT (batch) processing against our Secondary region -
Our teams are still working on bringing our systems back online. We'll continue to provide updates as they become available. We appreciate your understanding.
Posted Jul 19, 2024 - 02:40 CDT
Update
We believe a vendor supplied patch is causing issues on all webservers that handle traffic. Our team is working on uninstalling the impacted software but are having difficulties doing this. Our efforts are continuing to bringing our platform to a stable place. We will provide updates as they become available.
Posted Jul 19, 2024 - 01:06 CDT
Update
Our cloud provider is having issues with automatic restarts which hindered our progress in stabilizing our platform. Our team is working on getting services back up as quickly as possible. We'll post updates as products return to service.
Posted Jul 19, 2024 - 00:06 CDT
Update
We're aware of stability issues currently occurring. Our team is working to stabilize our platform. We'll continue to provide updates as they become available.
Posted Jul 18, 2024 - 23:48 CDT
Update
We are observing the following successful requests:
MFT (batch) - Success/failure emails may not be working currently, but file processing is operational. Please note MFT is running out of our DR location and some 3rd parties may not have our secondary IP whitelisted. Our team is working on shifting this back to main region currently.
Portal is still being worked by our team. We are monitoring for any outliers and will begin to address when our core services are back up.
Our teams are working continuously on this issue.
Posted Jul 18, 2024 - 23:29 CDT
Update
We are observing the following successful requests:
Transparent Gateway v1 Transparent Gateway v2 APIv1 APIV2 MFT (batch) - Success/failure emails may not be working currently, but file processing is operational
Iframe and Payment services v1 and v2, and Portal are still being worked by our team.
Our teams are working continuously on this issue.
Posted Jul 18, 2024 - 23:01 CDT
Update
Our teams are actively working on rolling back to our main region. We will continue to provide updates as efforts continue.
Posted Jul 18, 2024 - 22:20 CDT
Update
Our teams are continuing efforts bringing products as soon as possible. We will post updates as more products become available.
Currently stable - MFT (batch) is stable in DR scenario currently. Please utilize batch.tokenex.com or [redacted ip] to connect.
Transparent Gateway v2 is stable in DR scenario currently. Please utilize tgapi.tokenex.com or [redacted ip] to connect.
- We are aware of Transparent Gateway v2 /Hash possibly having issues - this will be prioritized accordingly. Our main focus remains on returning functionality to core products.
Posted Jul 18, 2024 - 21:21 CDT
Update
Our teams are continuing efforts -
MFT (batch) is stable in DR scenario currently. Please utilize batch.tokenex.com or [redacted ip] to connect.
Transparent Gateway v2 is stable in DR scenario currently. Please utilize tgapi.tokenex.com or [redacted ip] to connect.
Posted Jul 18, 2024 - 20:16 CDT
Update
Our cloud provider reported an outage in our active region. The decision was made to failover to our DR sites. Unfortunately, our disaster recovery plan utilizing multi-region configurations with our cloud provider is also currently unavailable. Our team is continuously working through products to regain functionality. We will continue to provide updates regularly.
MFT (batch) is stable in DR scenario currently.
Posted Jul 18, 2024 - 19:54 CDT
Update
We are continuing efforts on the DR pivot. The previously mentioned secondary IPs were an incomplete list. If you utilize IPs to connect with us - please switch to the secondary IPs provided below:
API api.tokenex.com Primary - [redacted ip] Secondary - [redacted ip] Port - [redacted port] Direction - Inbound
HTP (iFrame) htp.tokenex.com Primary - [redacted ip] Secondary - [redacted ip] Port - [redacted port] Direction - Inbound
MFT(batch/sftp) batch.tokenex.com Primary - [redacted ip] Secondary - [redacted ip] Port - [redacted port] Direction - Inbound
TGAPI tgapi.tokenex.com Primary - [redacted ip] Secondary - [redacted ip] Port - [redacted port] Direction - Inbound
Payment Services paymentservices.tokenex.com Primary - [redacted ip] Secondary - [redacted ip] Port - [redacted port] Direction - Inbound
Fraud Services fraudservices.tokenex.com Primary - [redacted ip] Secondary - [redacted ip] Port - [redacted port] Direction - Inbound
* If you are using a hostname (e.g., api.tokenex.com) to connect to our services then no changes are necessary.*
These products are still down as we work on pivoting to DR. We will provide updates here as products become available.
Posted Jul 18, 2024 - 19:13 CDT
Update
Our DR pivot efforts are continuing. We understand the impact and are working continuously to resolve the situation. We will continue to post updates regularly.
Posted Jul 18, 2024 - 18:39 CDT
Identified
Our DR pivot is still underway. If you are using a hostname (e.g., api.tokenex.com) to connect to our services then no changes are necessary. If you connect with an IP address, then the secondary IP address will need to be used. We will continue to provide updates regularly.
Production
API Primary - [redacted ip] Secondary - [redacted ip] Port - [redacted port]
We're aware of limited availability to our customer portal as well. We are initiating our protocol to switch to our DR sites. Please monitor this status page for further updates.
Posted Jul 18, 2024 - 17:53 CDT
Investigating
We're aware of an ongoing issue. You may experience limited availability. There is an issue with our cloud hosting provider that is causing latency for some of our products. We'll provide updates regularly.
Posted Jul 18, 2024 - 17:35 CDT
This incident affected: U.S. Data Center (iFrame, API v1, API v2, Transparent Gateway API v1, Transparent Gateway API v2, Payment Services v1 (Active Merchant), Payment Services v2, Batch/MFT).