Skip to content

Commit e3d6f9d

Browse files
committed
Algorithms update
1 parent ebd295b commit e3d6f9d

File tree

2 files changed

+198
-48
lines changed

2 files changed

+198
-48
lines changed
Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
## Terminology
2+
3+
- **Safety** in engineering is the prevention of harm through system design.
4+
5+
- An **operation** is any executable program or program fragment, from an integer addition to a whole application.
6+
7+
8+
- A **[safety property](https://en.wikipedia.org/wiki/Safety_and_liveness_properties)** is the *impossibility* of some occurrence when an operation is used correctly. For example, this function upholds the safety property that nothing is printed to the console:
9+
10+
```swift
11+
/// Returns `x`.
12+
///
13+
/// - Precondition: `x >= 0`
14+
func identity(_ x: Int) -> Int {
15+
if x < 0 { print("Precondition violated!") }
16+
return x
17+
}
18+
```
19+
To be a safety property, it must *compose*. That is, when any two operations *P* and *Q* uphold the property, so does *P* followed by *Q*. For example, freedom from data races is a safety property, but freedom from logical races is not, because when two consecutive non-racy mutations are composed into a larger mutation, another thread can observe the partially mutated state between the two steps.
20+
21+
- A **[liveness property](https://en.wikipedia.org/wiki/Safety_and_liveness_properties)** is the *guarantee* of some occurrence when an operation is used correctly. For example, this function upholds the liveness property that it eventually returns:
22+
23+
```swift
24+
/// Returns `x`.
25+
///
26+
/// - Precondition: `x >= 0`
27+
func identity2(_ x: Int) -> Int {
28+
while x < 0 { /* loop forever */ }
29+
return x
30+
}
31+
```
32+
33+
- An ***X* safe operation** upholds some safety property *X* **even if preconditions are violated**. [^qualification] For example, when `a` is an array, `a[0] = 3` never modifies a variable not mentioned in the expression, even if `a` is empty (which violates the precondition of `a[0]`). We might say that the operation is “expression-mutation safe.”
34+
35+
[^qualification]: note this important distinction—an operation can uphold the memory safety property but not be memory-safe by this definition, because the former depends on preconditions being satisfied but the latter does not.
36+
37+
- An ***X* safe language** is one where all primitive operations are *X safe*. It follows that all non-primitive operations—and all possible programs in the language—are *X safe*. A language subset, such as “Swift programs in which no identifier contains the substring `Unsafe` or `unsafe`,” can be considered a language.
38+
39+
- **Memory safety**: the property that invalid memory operations such as out-of-bounds accesses and use-after-free do not occur.
40+
41+
- **Type safety**: the property that an instance of one type is never accessed as an instance of another type.
42+
43+
- **Thread safety**: the property that a data race does not occur. Sometimes “thread safe” is used to mean that, additionally, deadlock does not occur. Freedom from deadlock can also be viewed as part of a liveness property guaranteeing forward progress.
44+
45+
- **Data race safety**: the property that a data race does not occur (explicitly excluding freedom from deadlock as a constraint).
46+
47+
- **Undefined behavior** is not bounded by any constraints and thus nullifies every safety property. An operation that can have undefined behavior, or a language that includes such an operation, is never *X* safe for any *X*.
48+
49+
Violations of memory safety, type safety, and data race safety have effects that can't be usefully described in terms of any portable programming language. For example, the effects of an out-of-bounds write can be understood when memory is viewed as a linear collection of bytes, but can't be described in terms of distinct variables and constants of many types. Therefore, in unsafe programming languages, these violations typically cause undefined behavior.[^java-data-race]
50+
51+
- A **safe operation** will never exhibit undefined behavior, even if preconditions are violated. Safety is often a consequence of type checking (you can't access `x.5` when `x` is a 2-element tuple), but sometimes runtime checks are needed, as when indexing a variable-length array. “Trapping” or otherwise stopping the program when preconditions are violated is one way to achieve safety.
52+
53+
- A **safe language** (such as Java or Haskell) has only safe operations, so all possible programs in the language are safe. The distinction is important because proving a safety property of arbitrary code is tedious and sometimes very difficult, unless—as with a safe language—all code is safe by construction.
54+
55+
- In practice, “**memory-safe language**” is synonymous with “safe language.” Since undefined behavior invalidates all guarantees (including memory safety), a memory-safe language can have no undefined behavior and is therefore a safe language. Because the behavior of a memory safety violation can't be defined at the language level, any language without undefined behavior must be memory safe.
56+
57+
- A **safe-by-default language** (such as Swift or Rust) contains a minority of unsafe operations that can be easily recognized by tooling and banned or flagged for extra scrutiny in code review. This arrangement provides unconditional safety in most code while allowing the direct use of primitive operations such as pointer dereferencing, without expensive validity checks. When unsafe operations are used correctly in the implementation details of safe abstractions, the vocabulary of safe operations grow, with little compromise to overall security. Safe-by-default languages are often referred to as “memory safe” despite the availability of operations that can compromise memory safety.
58+
59+
- The **safe subset of a safe-by-default language** is a safe language.
60+
61+
[^java-data-race]: Some languages, such as Java and JavaScript, define the behavior of data races, but in such a way as to be useless for most programming.
62+
63+
64+
----
65+
66+
In Lamport’s framework, safety is defined semantically—as a prefix‑closed set of behaviors—but this definition alone does not guarantee compositionality under functional composition. As Abadi and Lamport show in Composing Specifications, and as later clarified by Abadi and Plotkin’s work on refinement‑preserving transformations, safety properties become compositional only when the functions involved are themselves safety‑preserving. In other words, from the fact that a safety property p holds for f(x) and for g(x), nothing follows about p(f(g(x))) unless f and g each preserve p. This distinction—emphasized in surveys such as Freiling and Santen’s work on compositional reasoning—makes clear that prefix‑closure characterizes the semantic nature of safety, while congruence under composition requires an additional structural assumption about the operators acting on behaviors.
67+
68+
preserving.dvi
69+
https://lamport.org/pubs/abadi-preserving.pdf
70+
lamport.org
71+
72+
73+
99583.99626
74+
https://dlnext.acm.org/doi/pdf/10.1145/99583.99626
75+
dlnext.acm.org
76+
77+
78+
On the Composition of Compositional Reasoning | Springer Nature Link
79+
https://link.springer.com/chapter/10.1007/11786160_8
80+
link.springer.com
81+
82+
https://lamport.azurewebsites.net/pubs/abadi-composing.pdf

better-code/src/chapter-4-algorithms.md

Lines changed: 116 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -86,30 +86,53 @@ something, another to remove it, another to insert it somewhere else, and
8686
perhaps another to repair the structure afterward
8787
-->
8888

89-
## From Mechanism to Intent
90-
We can remove the selected shapes more efficiently. The trick is to walk two
91-
indices (`writeIndex` and `readIndex`) forward. The `writeIndex` finds the next
92-
selected item to be removed and the `readIndex` finds the subsequent unselected
93-
item. The shapes at the indices are swapped and we proceed until `readIndex`
94-
reaches the end. Finally, we trim the array from `writeIndex` to the end to remove
95-
all the selected elements.
89+
## Intent to Mechanism
90+
91+
We can remove the selected shapes more efficiently. The trick is to collect all
92+
of the unselected shapes at the start of the array, and then remove the
93+
remaining shapes all at once. We don't care about the shapes to be removed or
94+
their order.
95+
96+
To design an algorithm to collect the unselected shapes, we use a common
97+
technique of stating the postcondition and then seeing if there is a way to
98+
extend the postcondition to additional elements.
99+
100+
The desired postcondition for collecting the unselected shapes is that all the
101+
unselected shapes are in the range `0..<p` in their original order, where `p` is
102+
the count of unselected shapes.
103+
104+
Now, consider if this condition holds up to some point, `r`, instead of to the
105+
end of the array, where `r...` are the remaining shapes that may or may not be
106+
selected. To advance `r` we examine the shape at `shapes[r]`, if it is not
107+
selected, we need to add it to `0..<p`.
108+
109+
There are two ways[^two-ways] we could do that. The first is to copy the shape `shapes[p] =
110+
shapes[r]`, but there are costs to copying <!--Todo, run some benchmarks to
111+
determine the tradeoffs -->. The other option is to swap the element at
112+
`shapes[r]` with the element at `shapes[p]`.
113+
114+
[^two-ways]: Depending on your language, there may be additional options. We
115+
could relocate (move) the shapes. In Swift, we could do this with unsafe
116+
operations, but we would leave uninitialized memory at the end of the array,
117+
and there is no operation on the standard array to trim it.
118+
119+
Both approaches will preserve the relative
120+
order of the unselected shapes. Now we can write the code:
96121

97122
```swift
98123
/// Remove all selected shapes.
99124
func removeAllSelected(shapes: inout [Shape]) {
100-
var writeIndex = 0
125+
var p = 0
101126

102-
for readIndex in 0..<shapes.count {
103-
if !shapes[readIndex].isSelected {
104-
if writeIndex != readIndex {
105-
shapes.swapAt(writeIndex, readIndex)
106-
}
107-
writeIndex += 1
127+
for r in 0..<shapes.count {
128+
if !shapes[r].isSelected {
129+
shapes.swapAt(p, r)
130+
p += 1
108131
}
109132
}
110133

111134
// Remove all selected shapes from the end
112-
shapes.removeSubrange(writeIndex...)
135+
shapes.removeSubrange(p...)
113136
}
114137
```
115138

@@ -119,30 +142,32 @@ The act of verifying a loop requires us to establish:
119142

120143
- A _loop invariant_ that holds before each iteration (including the first), and
121144
at loop termination.
122-
- A _variant function_ that maps to a strictly decreasing value on each to prove
123-
termination
124-
- A _postcondition_ that is a statement of the state of the loop invariant
145+
- A _variant function_ a notional function that maps to a strictly decreasing
146+
value on each iteration to prove termination
147+
- A _postcondition_ that is a statement of the loop invariant
125148
at loop termination.
126149

127-
In our above we would have the following:
150+
Our loop invariant is almost completely covered by the description of our algorithm:
128151
- Loop invariant:
129-
- All unselected elements before `readIndex` have been moved to the front.
130-
- The region before `writeIndex` preserves the original relative order of
131-
unselected elements.
132-
- All elements in the range `writeIndex..<readIndex` are selected.
152+
- `0..<p` contains all unselected shapes in `0..<r`.
153+
- `p..<r` contains all selected shapes in `0..<r`
154+
- for all `i` in `0..<p`, the `i`th unselected element of the original array is in
155+
position `i` (i.e., unselected shapes are in their original
156+
order).
133157
- Variant function:
134-
- The difference between `shape.count` and `readIndex` is reduced by one at
158+
- The difference between `shape.count` and `r` is reduced by one at
135159
each iteration. The loop exits when the difference is zero.
136160
- Postcondition:
137-
- When the loop exists after `readIndex == shapes.count`, the loop
138-
invariant holds so all unselected elements appear in the prefix in their
139-
original relative order and all the selected elements are in `writeIndex..`.
161+
- When the loop exists, `r == shapes.count`, all unselected elements appear
162+
in `0..<p` in their original order and all the selected
163+
elements are in `p..`.
140164

141165
<!-- Should we formally state this proof? Maybe show it later in Dafny? -->
142166

143167
<!-- Do we want formal exercises? add the exercise answers in an appendix -->
168+
144169
As an exercise, look back at the two prior implementations of
145-
`removeAllSelected()` and prove to yourself that the loop is correct.
170+
`removeAllSelected()` and prove to yourself that the loops are correct.
146171

147172
As with contracts, the processes of proving to ourselves that loops are
148173
correct is something we do informally (hopefully) every time we write a loop.
@@ -154,42 +179,85 @@ The best way to avoid complexity of loops is to learn to identify and compose
154179
algorithms. The loop we just implemented is a permutation operation that
155180
partitions our shapes into unselected and selected subsequences. The relative
156181
order of the shapes in the unselected sequence is unchanged. This property is
157-
known as "stability" so this operation is a half-stable partition. The algorithm
182+
known as _stability_, so this operation is a half-stable partition. The algorithm
158183
is not specific to shapes so we can lift it out into a generic algorithm.
159184

160185
```swift
161-
/// Reorders the elements of the collection such that all the elements that match
162-
/// the given predicate are after all the elements that don’t match, preserving
163-
/// the relative order of the unmatched elements.
164-
///
165-
/// - Returns: The index of the first element that satisfies the predicate.
166-
func halfStablePartition<T>(
167-
_ array: inout [T],
168-
by belongsInSecondPartition: (T) -> Bool
169-
) -> Int {
170-
var writeIndex = 0
171-
172-
for readIndex in 0..<array.count {
173-
if !belongsInSecondPartition(array[readIndex]) {
174-
if writeIndex != readIndex {
175-
array.swapAt(writeIndex, readIndex)
186+
extension MutableCollection {
187+
/// Reorders the elements of the collection such that all the elements that match
188+
/// the given predicate are after all the elements that don’t match, preserving
189+
/// the order of the unmatched elements. Returning the index of the first
190+
/// element that satisfies the predicate.
191+
mutating func halfStablePartition(
192+
by belongsInSecondPartition: (Element) -> Bool
193+
) -> Index {
194+
var p = startIndex
195+
196+
for r in indices {
197+
if !belongsInSecondPartition(self[r]) {
198+
swapAt(p, r)
199+
formIndex(after: &p)
176200
}
177-
writeIndex += 1
178201
}
202+
203+
return p
179204
}
180-
181-
return writeIndex
182205
}
183206
```
184207

185208
Given `halfStablePartition()` we can rewrite `removeAllSelected()`.
186209

187210
```swift
188211
func removeAllSelected(shapes: inout [Shape]) {
189-
shapes.removeSubrange(halfStablePartition(&shapes, by: { $0.isSelected })...)
212+
shapes.removeSubrange(shapes.halfStablePartition(by: { $0.isSelected })...)
190213
}
191214
```
192215

216+
Although we can't do better than linear time, our implementation of
217+
`halfStablePartition()` is doing unnecessary work by calling swap when `p == r`.
218+
As an exercise, before entering the loop, find the first point where a swap will
219+
be required, prove the new implementation is correct. Create a benchmark to
220+
compare the performance of the two implementations.
221+
222+
<!-- provide answer in appendix -->
223+
224+
Often, we view _efficiency_ as the upper-bound (big-_O_) of how an algorithm
225+
scales in the worst case. Scaling is important, but so are metrics like how much
226+
wall clock time the operation takes or how much memory the operation consumes.
227+
In the software industry if a competitors approach takes half the time or runs
228+
in production at half the energy costs, that could be a significant advantage.
229+
230+
We define _efficiency of an operation_ as the minimization of resource the
231+
operation uses to calculate a result. The resources include:
232+
233+
- time
234+
- memory
235+
- energy
236+
- computational hardware
237+
238+
Because memory access is slow, energy consumption is largely determined by the
239+
amount of time an operation takes, and computational hardware is often
240+
underutilized, we prioritize optimizing time. But as we will see in the
241+
Concurrency chapter, balancing all of these resources is critical for an
242+
efficient system design.
243+
244+
When designing algorithms it is important to have a rough sense of the cost of
245+
primitive operations to guide the design. Some approximate numbers by order of
246+
magnitude[^operation-costs]:
247+
248+
| Cycles | Operations |
249+
|---|---|
250+
| 10^0 | basic register operation (add, mul, or), memory write, predicted branch, L1 read |
251+
| 10^1 | L2 and L3 cache read, branch misprediction, division, atomic operations, function call |
252+
| 10^2 | main memory read |
253+
| 10^3 | Kernel call, thread context switch (direct costs), exception thrown and caught |
254+
| 10^4 | Thread context switch (including cache invalidation) |
255+
256+
[^operation costs]: [_Infographics: Operation Costs in CPU Clock
257+
Cycles_](http://ithare.com/infographics-operation-costs-in-cpu-clock-cycles/)
258+
259+
<!-- This section needs to close with a reference to removeWhere -->
260+
193261
A rotation expresses “move this range here.” A stable partition expresses
194262
“collect the elements that satisfy this predicate.” A slide is a composition of
195263
rotations. A gather is a composition of stable partitions.

0 commit comments

Comments
 (0)