WFS UDP BigData

Unique Data Platform — WFS BigData platform built on PySpark, Airflow and SQL. Available in two topologies: Hybrid (PaaS + SaaS) or Databricks (SaaS). Unified structure, high performance and WFSLib cutting dev time by 70%.

PySpark · Airflow · SQL 10× faster 40-70% savings
Arquitetura WFS UDP — fluxo de dados Bronze/Silver/Gold
WFS UDP Architecture Medallion flow · Bronze · Silver · Gold
What is WFS UDP

Unique Data Platform — Unified Data Structure

Six pillars that replace dozens of loose tools with a cohesive, governed, scale-ready platform.

Unified Structure

Defines projects, storage areas and AI-powered Data Catalog. Everything organized, documented and discoverable.

Complete History

Full history preservation with Delta Lake versioning. Rollback any data, anytime.

High Performance

Scalability, volume and speed without limits. Spark, Synapse Serverless, Delta Lake — Azure Cloud, on-premises or Databricks.

WFSLib

Proprietary library with ready extraction patterns. Cuts 70% of development, standardizes quality.

Data Portal

Web environment for manual file uploads with automatic validation. Business areas send data without IT.

WFS Portal Insights Live

Native integration with WFS Portal Insights via Live Connection. Data always up-to-date, no schedulers.

Stack & Topologies

PySpark · Airflow · SQL — in two topologies

Pick the model that fits your maturity and current ecosystem. WFSLib and governance are the same in both.

Topology 1 · Lower cost

Hybrid (PaaS + SaaS)

Combination of Azure-managed PaaS services with SaaS layers — Spark Cluster (PaaS), Data Lake Gen2, Synapse Serverless and Airflow. Lowest-cost topology, ideal for operational efficiency and flexibility.

  • Lowest infrastructure cost
  • Spark Cluster (Azure PaaS)
  • Data Lake Gen2 (storage)
  • Synapse Serverless (SQL)
  • Airflow orchestration
  • Spark Streaming · Quality · Rollback
Topology 2

Databricks (SaaS)

UDP runs natively on Databricks Lakehouse — leverages Unity Catalog, Photon Engine and Databricks' unified Spark stack. Ideal for companies already on or standardizing on Databricks.

  • Databricks Lakehouse (SaaS)
  • Unity Catalog for governance
  • High-performance Photon Engine
  • Native Databricks Workflows
Architecture

Medallion flow: Bronze · Silver · Gold

Data from TOTVS, VTEX, APIs, databases and Portal enter via Spark Streaming in real time and progress through governed layers to consumption. Customizable structure for other database standards and new data sources as your business needs evolve.

Arquitetura WFS UDP

Customizable for any scenario

The UDP architecture can be customized for other database standards (Oracle, SQL Server, PostgreSQL, MySQL, MongoDB, DB2, etc.) and any input source — SaaS, ERPs, CRMs, REST/GraphQL APIs, files, messaging (Kafka, EventHub), IoT and more. WFSLib eases the extension without losing governance.

End-to-end journey

End-to-end data engineering

From the first data at the source to the final Data Science insight — we cover all 5 stages in one platform, with governance and WFSLib.

Engenharia de dados ponta a ponta — WFS UDP BigData
Quantified results

Real benefits, measured in real projects

40-70%

Cost Reduction

Operational and infrastructure

10×

Faster

Query performance

50%

Less Dev

Dev time with WFSLib

100%

Built-in Quality

DEV→QA→PRD with rollback

Proven cases

Production results

VOLVO

−45%

Manufacturing
Unified quality view

SHELL

Energy
Operational big data

OBDI

−60%

Logistics
Real-time telemetry

Implementation roadmap

From diagnosis to operation in ~9 months

Three progressive phases with break-even in 12 to 15 months.

01

Foundation

2-3 months

  • ✓ Cloud infrastructure setup
  • ✓ Migration of critical pipelines
  • ✓ Team training
  • ✓ DEV/QA environment
02

Expansion

3-4 months

  • ✓ Full e-commerce migration
  • ✓ Real-time implementation
  • ✓ Data Portal for business
  • ✓ Operational PRD environment
03

Optimization

2-3 months

  • ✓ AI/ML implementation
  • ✓ Advanced automation
  • ✓ Executive dashboards
  • ✓ Complete governance
Team transition eased

Short learning curve, expertise preserved

✓ Expertise preserved

All the team's Python and Airflow knowledge is leveraged — UDP uses the same market tools.

✓ Complete training

Spark and UDP architecture training by WFS, with real cases and practical mentoring.

✓ Fast curve

PySpark is similar to Python — accelerated learning and productivity from the first weeks.

Multi-area governance

Any area of the company can use it — with full control

✓ Single source

All data respects the same source of truth. No duplication, no divergence, no "which number is correct?".

✓ Granular controls

Delta Lake allows releasing data to any area without failures. Permissions by row, column, project and user.

✓ Complete audit

Total history of access and changes by area. Meets compliance, LGPD/GDPR and internal audit.

✓ Quality pipeline

DEV → QA → PRD with automatic validation and rollback. No bad data reaches production.

Next steps

How to start

01

Executive approval

Definition of a dedicated team and resources for the transformation.

02

POC with real data

UDP validation using your own data, in an isolated environment.

03

TCO analysis

Total Cost of Ownership study with a customized migration plan.

Ready for the digital transformation of your data architecture?

30-minute conversation with a WFS architect. We present UDP in any mode (Hybrid PaaS+SaaS or Databricks SaaS), discuss your scenario and propose a POC.

Schedule POC
Frequently asked questions

About WFS UDP BigData

What is WFS UDP BigData?

WFS UDP (Unique Data Platform) BigData is WFS's proprietary platform for building data lakes and lakehouses at enterprise scale. It combines a medallion architecture (bronze/silver/gold) on Spark, Delta Lake and Synapse with WFSLib — a proprietary library that cuts pipeline development time by up to 70% and delivers 10× the performance of traditional implementations.

What success cases does UDP BigData have?

In production at companies such as Volvo, Shell, OBDI and Britânia, processing hundreds of TB per day. Cases include unifying manufacturing data, integrating Brazilian ERPs (TOTVS, SAP) with cloud analytics, and modernizing legacy DWs (Teradata, Netezza) to cloud-native architecture.

Which cloud does UDP BigData run on?

Full support for Microsoft Azure (Synapse, Fabric, Databricks, ADLS Gen2), AWS (EMR, Glue, Redshift, S3), Google Cloud (Dataproc, BigQuery, GCS) and on-premises environments (Cloudera, Databricks on-prem, Hadoop). Also supports hybrid and multi-cloud architectures for data sovereignty cases.

What is WFSLib and why does it cut 70% of development?

WFSLib is a proprietary library with ready-made components for common pipeline patterns: incremental ingestion, deduplication, data quality, optimized partitioning, slowly changing dimensions, change data capture, observability. Teams using WFSLib ship new pipelines in days instead of weeks, with consistent quality and technical standards.

Do you migrate legacy DWs to UDP BigData?

Yes. Migration projects from Teradata, Netezza, Oracle DW to modern lakehouses (Databricks, Snowflake, Synapse) with stored-procedure refactoring, data-parity validation, phased cutover and training. Typical cases deliver the first domain in 8-12 weeks and full migration in 6-12 months.