-
What type of deployment does DataForge support within Amazon Web Services, Microsoft Azure, or Google Cloud Platform?
- The standard product offering for DataForge Cloud is a SaaS-based model where the customer hosts datalake storage and compute resources. DataForge hosts all other services and infrastructure. DataForge also offers a private cloud deployment where all services and infrastructure are hosted within the customer cloud environment.
-
Does DataForge own and manage the Infrastructure that supports our version of the product?
- In DataForge Cloud, storage and compute are managed through DataForge, but owned by the customer in third-party services (AWS/Azure/GCP/Databricks). All other infrastructure is owned and managed by DataForge. The customer signs up for various services, including AWS, Azure, or GCP as part of the initial deployment and owns these services. In Private Cloud Deployments, DataForge does not own the infrastructure and monitoring within a customer environment. DataForge provides the infrastructure code updates to maintain and keep DataForge running in the form of new versions of the platform. The customer owns the process of staying up to date with the latest DataForge version releases.
-
What happens if there is an infrastructure issue and it is unclear how to resolve it?
- If you find yourself in a situation where something is broken or not working correctly, please reach out to DataForge support. We will triage all issues and if it is deemed an issue that needs to be resolved by the infrastructure service provider (i.e. AWS, Azure, Databricks) we will communicate that and ask that the customer open a support request with the appropriate technology vendor. If the issue is something the DataForge team manages, we will provide guidance on how to resolve the issue.
-
Is it possible to customize the networking and infrastructure before or after the initial DataForge deployment?
- Yes, changes to networking and infrastructure can be made to the customer environment by the customer or a third-party provider. It is highly recommended to make any changes to a non-production environment to test that all data flows are working correctly before any changes are promoted to a production environment. If issues arise due to custom infrastructure changes, the DataForge Support team will do its best to help provide guidance, but ultimately the customer owns these changes and will need to resolve the issues or backtrack the changes until they are able to be resolved.
-
Is it possible to restart only certain services that DataForge is running for Private Cloud deployments?
- Yes, the DataForge platform is built in a containerized fashion and depending on the need, different services can be restarted or scaled up/down. For more information on restarting services for Private Cloud deployments, please visit the help center guide for AWS Containers or Azure Resources (requires being signed in to the Help Center).
-
How can I check my infrastructure costs?
- Infrastructure costs will be shown and distributed by each service provider that an account is set up for. Costs will be split into AWS or Azure or GCP, Databricks, and DataForge. Utilization and cost can be accessed through each UI. DataForge usage can be queried via the metadata database in Databricks.
-
Do we need to provide the DataForge team access to our infrastructure in order to be a customer and get support for issues?
- Although it is not required to provide access to the DataForge team to use the product, it is highly beneficial to share access as it significantly speeds up time to resolution for support issues. DataForge engineers/architects can provide guidance on best practices depending on what is visible to us as we troubleshoot issues or questions.
-
Is there built-in alerting and monitoring for AWS or Azure or GCP infrastructure services?
- For process alerting with DataForge (failed vs. successful jobs), email alerts or AWS SNS can be set up. By default, there is no infrastructure alerting set up for customers who deploy DataForge. Monitoring can be set up through the cloud provider but is the responsibility of the customer.
Updated