9.0 Maintenance Updates

This article lists maintenance fixes and updates for supported DataForge versions.

DataForge Version 9.0

• May 30, 2025

  • Add com.dataforge as maven alternative for SDK

    • Deployed maven library to replace JAR

  • SaaS: Change Logout button to drop-down with options

    • Updated logout option in user menu drop-down

  • Force all cluster configurations to move to latest DBR (custom cluster configrations)

    • All cluster configurations are upgraded to the correct DBR during upgrade deployment

  • Connection metadata search and checkboxes

    • Connection metadata checkbox selection is retained as the search is cleared and changed. Multiselect is possible with shift + click.

  • Databricks Version Upgrade to 15.4 LTS

    • Update to DBR 15.4 LTS

  • Connection data dictionary build, import, reference, UI, etc.

    • New data dictionary available in connection for table and column level. Defines descriptions, categories, keys, and fk tables. Export/import from connection settings page.

  • Dark mode UI

    • Dark Mode available through new user menu. Light/dark mode can be switched per profile.

  • Add "category" for connection_table_column and rules to help automate output creation

    • Category is included in column level data dictionary for connections. Category also exists for raw attributes and rules. Drives Talos auto detection and config creation.

  • Talos rule creation

    • Use Talos to create new rules with prompts like "Create rule in x source that does y using source z" or "Calculate x in source y"

  • Talos Output 2.0

    • Use Talos to create outputs using natural language processing using the create output workflow with prompts like "Create output using matched columns" or "Create output using x, y, and z sources"

  • Allow import of connection tables PK/FK via data ditionary

    • Identify key columns and foreign tables in the connection column level data dictionary. Use fk, pk, pk+fk to identify the column type. Use schema_name.table_name to identify the fk table.

  • Enable s_key on full refresh

    • Use the Key Columns parameter in source settings with Full refresh to identify key columns and check for uniqueness during CDC processing. S_key column will contain the result of the key columns identified.

  • Recommend primary keys and relations

    • Talos can recommend relations between sources after running data profile on sources twice. Talos uses the result of the of the data profile to match values across sources and use column name matching to identify potential relations.

  • Relation template related source cardinality is not updated when expression is changed

    • Related source cardinality is now shown in relation templates in addition to [This] source cardinality.

  • Enable loose matching of column names for upsert_output

    • Allows loose matching of source column names when creating or updating output

  • Update Terraform to not wipe out key vault access policies each time it applies

    • Add a terraform variable to retain access policies in keyvault rather than clearing them. Use variable name "developer_object_ids " and value of ["<object-id>"] where it is comma separated with multiple

Updated May 07, 2025 15:34