Distributed Processing

Data Engineeringsystem

First met at Engineer6h

In simple words

Spark and its relatives. Partitions, shuffles, and the skew that leaves one task running for six hours.

The fuller explanation is not written yet

This term has its meaning, its place on the map and its sources, and they are accurate. The longer write-up is still to come, and this page will say so until it does rather than padding the gap.

Where does this sit on your route?

The free assessment places you on the same map and names which terms stand between you and the role you want.

Take the free assessment

See it in context

The Atlas shows this term with everything that leads into it and everything that follows, as one picture.

Open the map