Unit 8 · Lesson 5

Distributed Algorithms

Parallel computing, speedup, and the limits of adding more processors.

Warm-Up

Faster With Help

What is a task that you can complete faster if you get other people to help?

What is the most number of people you would want helping you, and why?

Activity — Parallel Algorithms and Speedup

Get into groups and grab your blocks.

  • Form groups of 3 or 4 students and move to the tables.
  • Each group should have a tub of blocks. The blocks come in 9 different shapes.
  • Your goal in every challenge is to sort the blocks into 9 neat piles by shape.
  • You will run three timed challenges: one person sorts, then two, then your whole group.
Colorful 3D building blocks of different shapes: rectangles, arches, cylinders, and triangles
Activity — Challenge One

One Person Sort

Shake up the tub to mix the blocks.

When the timer starts, one person sorts all the blocks into piles by shape as fast as possible.

Time stops when the blocks are sorted into 9 neat piles.

Run this once for each person in your group. Keep track of each time and record the best.

00:00

Record the best time in your group.

Activity — Challenge Two

Two Person Sort

Shake up the tub to mix the blocks.

When the timer starts, two people work together to sort the blocks into piles by shape.

Time stops when the blocks are sorted into 9 neat piles.

Run this once for each possible pair in your group. Record the best pair time.

00:00

Record the best time in your group.

Activity — Challenge Three

Full Group Sort

Shake up the tub to mix the blocks.

When the timer starts, your entire group works together to sort the blocks into piles by shape.

Time stops when the blocks are sorted into 9 neat piles.

Run this once. Everyone participates at the same time.

00:00

Record your group's time.

Concept

Sequential vs. Parallel

Sequential

Steps are performed in order, one at a time. The next step cannot begin until the current one is done.

Sequential flow diagram: three tasks connected by downward arrows

Parallel

Some steps are performed at the same time, with multiple workers acting simultaneously.

Parallel flow diagram: tasks split across multiple workers and reconverge
Discussion

Reflect on Your Algorithm

Question 1

What portions of your algorithms for Challenges 2 and 3 were parallel? Describe specifically what two or more people were doing at the same time.

Question 2

What made things complicated or slowed you down during the parallel portions? Were there moments when only one person could act at a time?

Concept

Speedup

Speedup measures how much faster a parallel solution runs compared to a sequential one, using the same amount of work.

Formula
Speedup =
sequential time parallel time

Example

Sequential flow diagram with one path of work done step by step

Sequential

60 seconds

Parallel flow diagram with multiple branches of work happening at the same time

Parallel

40 seconds

60 ÷ 40 = 1.5

The speedup of this parallel solution is 1.5.

Activity — Calculate Your Speedup

Your Group's Speedup

Sequential flow diagram with one path of work done step by step

Sequential

Your solo time

Parallel flow diagram with multiple branches of work happening at the same time

Parallel

Your team time

solo time ÷ team time = speedup
Prompt

What was your group's speedup in Challenge 2 (two-person sort)? Divide your best solo time by your best pair time.

Prompt

What was your group's speedup in Challenge 3 (full group)? Is there anything surprising about the result?

Concept — It's Not Just You!

Speedup always hits a ceiling.

  • Speedup is never equal to the number of processors added.
  • Some portions of any algorithm cannot be parallelized and must still run one step at a time.
  • Each additional processor helps a little less than the one before it.
  • Eventually speedup reaches a hard limit, no matter how many processors you add. This relationship is described by Amdahl's Law.
Amdahl's Law graph: speedup vs number of processors for 50%, 75%, 90%, and 95% parallel portions. Each curve flattens and approaches a fixed ceiling.
Video — Distributed Computing

As You Watch

Write down answers to these questions while watching the video on the next slide.

Question 1

Why is the type of computing shown in this video considered "distributed" computing?

Question 2

Why is distributed computing a good fit for this particular problem? What would make solving it sequentially impractical?

Distributed Computing — Video

Watch Together

The video will play when SyncDeck activates. Keep your notes ready.

Wrap-Up — Vocabulary

Key Terms

Sequential Computing
Programs run in order, one command at a time. Each step must finish before the next begins.
Parallel Computing
A program is broken into smaller pieces, and some of those pieces run simultaneously on multiple processors.
Distributed Computing
A program is run by multiple separate devices working together over a network to solve a problem too large for one machine.
Speedup
The time to complete a task sequentially divided by the time to complete the same task in parallel. Speedup is always less than the number of processors added.
Discussion Question

Parallel, Distributed, or Both?

Is the block-sorting activity from today a parallel algorithm, a distributed algorithm, or both? Use the definitions we discussed and evidence from the activity to explain your answer.

Exit Questions

Check Your Understanding

Free Response

Could the block-sorting activity be turned into a distributed algorithm? Describe what would need to change. If you think it could not work as distributed, explain why not.

Free Response

Based on today's activities, what are the pros and cons of parallel and distributed computing?

AP-Style Multiple Choice

Which of the following best explains why the speedup from parallel computing is never equal to the number of processors added?

  • A Parallel programs require more memory, which slows down each processor.
  • B Some portions of any program cannot be parallelized and must run sequentially.
  • C Processors must take turns accessing data, which is slower than sequential.
  • D Parallel programs are harder to write, so programmers leave some parts sequential.