International Retail Apparel Company: Optimized Teradata to Snowflake Migration

Organization

Our client is a renowned retail apparel chain headquartered in Texas, operating over 1,500 stores across the US and Canada. As a leader in the fashion retail industry, the company has strategically acquired several prominent market players as subsidiaries, further expanding its market presence. Backed by a robust supply chain and warehouse network, their team of approximately 22,500 employees provides a high-touch, personalized shopping experience, driving loyalty across all their brands.

Challenge

Migrating to the cloud was a key component of the client’s digital transformation strategy. The primary objective was to enhance data accessibility, enable elastic scalability, and reduce IT operational expenses. At the time, their Teradata data warehouse platform was nearing full capacity and approaching its end of life. Expanding the on-premise Teradata infrastructure would have incurred significant additional costs, making cloud migration the most viable option as part of the overall technology overhaul.

The decision to transition from Teradata to the Snowflake Cloud Data Platform was driven by an in-depth analysis of several key factors, including cost efficiency, ease of workload management, seamless migration, elastic scalability, user-friendliness, and the ability to effectively replicate and recover data.

Solution

With over 20 years of expertise in data warehouse implementations and migrations, XTIVIA played a pivotal role in the seamless migration from Teradata to Snowflake. Collaborating as part of a multi-vendor team, XTIVIA leveraged its data warehouse migration and validation frameworks to ensure a smooth transition, meeting the project’s timeline and objectives.

• Executed the initial data migration from Teradata to Snowflake
• Migrated DataStage jobs from Teradata to Snowflake
• Adapted the existing Groovy SQL code generator tool to produce Snowflake-compatible ETL SQL
• Compared Snowflake SQL with the original Teradata SQL, addressing discrepancies and applying necessary fixes
• Validated and provided feedback on the code generated by other vendors
• Scheduled Snowflake ETLs using the UC4 scheduler and ran them in parallel with existing Teradata UC4 jobs
• Conducted parallel load testing and synchronized data between Teradata and Snowflake
• Monitored Snowflake performance under existing Teradata workloads, optimizing performance where needed
• Supported the validation of historical data between Teradata and Snowflake
• Reviewed test cases and results from other vendors, providing feedback
• Deployed Snowflake code (DDLs, ETL, UC4 Objects, UC4 Schedules) to the UAT environment
• Monitored daily UAT loads in Snowflake, addressing issues as they arose
• Resolved UAT data issues and coordinated with other vendors to address critical code issues
• Migrated code (DDLs, ETL, UC4 Objects, UC4 Schedules) to the production environment
• Monitored production loads (Snowflake and Teradata in parallel), clarified any differences, and resolved minor data synchronization issues
• Developed additional ETLs for new summary tables in Snowflake to migrate complex transformation logic from MicroStrategy to Snowflake

Business Result

• A stable and reliable data warehouse platform
• High-performance ETL flows
• Enhanced workload and compute resource management, resulting in improved data availability and accessibility
• Significant reduction in administrative costs (eliminating the need for query tuning, data distribution, statistics collection, index management, maintenance windows, disk failure resolution, patching, upgrades, capacity monitoring, report reviews, workload analysis, and capacity planning)
• Lowered storage and compute costs (only pay for computing resources used)
• Increased visibility into cost and resource utilization
• Elastic scalability with auto-suspension of computing resources based on defined rules
• Zero-copy cloning of production data for use in non-production environments

KEYWORDS

Business Intelligence, Data Warehousing, Data Warehouse Migration, Data Migration, Data Integration, OLAP, Daas (Data as Service), Snowflake, Enterprise Data Warehouse, Cloud Data Lake, Cloud Data Lakehouse

SOFTWARE

Snowflake, Teradata, Python, SUSE Linux, InfoSphere DataStage, Eclipse, UC4

XTIVIA needs the contact information you provide to us to contact you about our products and services. You may unsubscribe from these communications at anytime, read our Privacy Policy here.