This article lists maintenance updates for supported DataForge versions.
DataForge Version 8.0
- September 25, 2024 (8.0.4)
- Cleanup fails to update meta.process fix
- Parametrize max relation hops to prevent API crashes
- Remove spark.sql.decimalOperations.allowPrecisionLoss from default cluster spark configs
- Ingestion fails when struct has fewer fields than previously
- September 3, 2024 (8.0.3)
- Add All Source Columns after columns are added/mapped breaks output positions (writes incorrect data in columns)
- Cleanup is failing with error process_history_v2
- Enrichment parser timeout fix
- Keep current rule using Self-relation with multiple relation hops causes query gen error
- Add customer billing portal button to accounts page
- Add "reselect cloud provider" option to signup flow after selecting cloud and before creating workspace
- Send email when new customer backs out of payment entry
- Add subscription type to signup UI
- Add subscription limit tracking & enforcement for Plus subscription edition
- August 19, 2024 (8.0.2)
- Rule expression validator fails when expression uses same parameter twice or more
- Unchecking unique flag on rule does not remove uniqueness rule and has to be deleted in postgres
- Sparky ingest isn't adding trust server certificate (and maybe more) to connection correctly
- Prevent endless deployment loop + send notification when container crashes repeatedly
- Add s3 endpoint to SaaS environments to avoid costs for deployment artifact download
- Virtual or delta output fail query gen when Timestamp attr is mapped to Date output column (corrected to invalid mapping - not allowed anymore)
- Removed other invalid output type mapping options. Refer to schema evolution matrix in source settings documentation for valid combinations.
- August 2, 2024 (8.0.1)
- Workflow queue put wait on a source for the wrong output
- Output with Array/Struct column fails all channel outputs that aren't mapped to the array
- Changing schedule on custom refresh source prompts Reset All CDC required
- Enrichment failed due to raw attribute "view" is missing in last input
- Large complex attribute schema breaks data normalization
- Import process should not validate inactive rules have parameters since we don't save them
- Blank column shows after adding long source to output
- Azure saas workspaces do not have default cluster config node type and saving \(default\) throws error with both cloud node types
- Double type is invalid for sql server output
- Custom notebook process end Not sending to correct api route in SaaS
- Output column can't be remapped to decimal from double
- Update enrichment datatype and datatype\_schema to account for cast\_datatype
- Add infra permissions so core can update deployment task definition/container image
- July 3, 2024 (8.0.0)
- Agent is not applying compression to avro files
- Having a space in Output column name for file output throwing PARSE_SYNTAX_ERROR error
- Changing refresh type full to key leaves existing relations and rules with incorrect cardinality
- Raw attribute normalization does not work properly for s_output_source_id
- Reset all CDC breaks if run immediately after import of compound key keyed source
- Switching from scheduled to watcher doesn't clear out record in ingestion queue
- Relation Convert to Template tooltip with Primary Flag
- Refresh browser or connection metadata view when switching projects
- Sensitive connection parameters with ( in the param value can't be extracted and print in notebook runs
- Fix scroll bar/buttons on Rule editor page
- Agent logs don't scroll/load past certain number of records
- Processing page filters spread really far apart on bigger screen
- Import doesn't restart if core goes through reboot
- Data View sort column doesn't allow sort descending anymore (click column sort twice)
- Manual CDC failed with error message- Cannot convert Avro field 'linetotal'
-
Getting group message "No tokens defined for any sources within cloned group" when sources have tokens
- Rule Template Applying - Updated By and Updated Date are wrong
- Cleanup causes issues when it runs alongside a virtual output when channel has no data
- Trim_string_data parameter is ignored in Sparky Ingest
- Select Action / Remove Linked sources is not visible on rule and relation template linked sources page
- Importing same project twice showing relation object changed
-
Add scrollbar on Account Management page
-
Opening project import url opens imports from correct project but wrong project name dropdown
- Connection dropdown doesn't load on source settings page randomly
- Changing file type is showing parse reset message - should show message that file type can't be switched
- None.get error massage during connection saving for invalid json
- Error on ingestion for connection metadata created sources
- Token list is broken in the test source on the Relation Template page
- Project import failing on "unable to map relations"
-
Import failed : ERROR: too few arguments for format()
- Schedule not picked up by Core after cloning sources
- Complex data types is failing during refresh
- Rule expression validation fails with AGG(...) + [..].x pattern
- Remove Recalculate changed
- The save output mapping button should not be available to click when the source is not selected
- After import, "next ingestion" should not be populated on all sources page
- Wrong updated userid on the source setting page
Updated