Вы здесь

Новости LessWrong.com

Подписка на Лента Новости LessWrong.com Новости LessWrong.com
A community blog devoted to refining the art of rationality
Обновлено: 50 секунд назад

Pattern's Shortform Feed

31 мая, 2019 - 00:21
Published on May 30, 2019 9:21 PM UTC


Uncertainty versus fuzziness versus extrapolation desiderata

30 мая, 2019 - 16:52
Published on May 30, 2019 1:52 PM UTC

.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')}

I proposed a way around Goodhart's curse. Essentially this reduces to properly accounting all of our uncertainty about our values, including some meta-uncertainty about whether we've properly accounted for all our uncertainty.

Wei Dai had some questions about the approach, pointing out that it seemed to have a similar problem as corrigibility: once the AI has resolved all uncertainty about our values, then there's nothing left. I responded by talking about fuzziness rather than uncertainty.

Resolving ambiguity, sharply or fuzzily

We have a human H, who hasn't yet dedicated any real thought to population ethics. We run a hundred "reasonable" simulations where we introduce H to population ethics, varying the presentation a bit, and ultimately ask for their opinion.

In 45 of these runs, they endorsed total utilitarianism, in 15 of them, they endorsed average utilitarianism, and in 40 of them, they endorsed some compromise system (say the one I suggested here).

That's it. There is no more uncertainty; we know everything there is to know about H's potential opinions on population ethics. What we do with this information - how we define H's "actual" opinion - is up to us (neglecting, for the moment, the issue of H's meta-preferences, which likely suffer from a similar type of ambiguity).

We could round these preferences to "total utilitarianism". That would be the sharpest option.

We could normalise those three utility functions, then add them with the 45-15-40 relative weights.

Or we could do a similar normalisation, but, mindful of fragility of value, we could either move the major options to equal weights 1-1-1, or stick with 45-15-40 but use some smooth minimum on the combination. These would be the more fuzzy choices.

All of these options are valid, given that we haven't defined any way of resolving ambiguous situations like that. And note that fuzziness looks a lot like uncertainty, in that a high fuzziness mix looks like what you'd have as utility function if you were very uncertain. But, unlike uncertainty, knowing more information doesn't "resolve" this fuzziness. That's why Jessica's critique of corrigibility doesn't apply to this situation.

(And note also that we could introduce fuzziness for different reasons - we could believe that this a genuinely good way of resolving competing values, or it could be to cover uncertainty that would be too dangerous to have the AI resolve, or we could introduce it to avoid potential Goodhart problems, without believing that the fuzziness is "real").

Fuzziness and choices in extrapolating concepts

The picture where we have 45-15-40 weights on well-defined moral theories, is not a realistic starting point for establishing human values. We humans start mainly with partial preferences, or just lists of example of correct and incorrect behaviours in a narrow span of circumstance.

Extrapolating from these examples to a weighting on moral theories is a process that is entirely under human control. We decide how to do so, thus incorporating our meta-preference implicitly in the process and its outcome.

Extrapolating dogs and cats and other things

Consider the supervised learning task of separating photos of dogs from photos of non-dogs. We hand the neural net a bunch of labelled photos, and tell it to go to work. It now has to draw a conceptual boundary around "dog".

What is the AI's concept of "dog" ultimately grounded on? It's obviously not just on the specific photos we handed it - that way lies overfitting and madness.

But nor can we generate every possible set of pixels and have a human label them as dog or non-dog. Take for example the following image:

That, apparently, is a cat, but I've checked with people at the FHI and we consistently mis-identified it. However, a sufficiently smart AI might be able to detect some implicit cat-like features that aren't salient to us, and correctly label it as non-dog.

Thus, in order to correctly identify the term "dog", defined by human labelling, the AI has to disagree with... human labelling. There are more egregious non-dogs that could get labelled as "dogs", such as a photo of a close friend with a sign that says "Help! they'll let me go if you label this image as a dog".

Human choices in image recognition boundaries

When we program a neural net to classify dogs, we make a lot of choices - the size of the neural net, activation functions and other hyper-parameters, the size and contents of the training, test, and validation sets, whether to tweak the network after the first run, whether to publish the results or bury them, or so on.

Some of these choice can be seen as exactly the "fuzziness" which I defined above - some options determine whether the boundary is drawn tightly or loosely around the examples of "dog", and whether ambiguous options are pushed to one category or allowed to remain ambiguous. But some of these choices - such as methods for avoiding sampling biases or adversarial learning example of a panda as a gibbon - are much more complicated than just "sharp versus fuzzy". I'll call these choices "extrapolation choices", as they determine how the AI extrapolates from the example we have given it.

Human choices in preference recognition boundaries

The same will apply to AIs estimating human preferences. So we have three types of things here:

  • Uncertainty: this is when the AI is ignorant about something in the world. Can be resolved by further knowledge.
  • Fuzziness: this how the AI resolves ambiguity between preference-relevant categories. It can look like uncertainty, but is actually an extrapolation choice, and can't be resolved by further knowledge.
  • Extrapolation desiderata: extrapolation choices are what need to me made to construct a full classification or preference function from underdefined examples. Extrapolation desiderata are the formal and informal properties that we would want these extrapolation choices to have.

So when I wrote that to avoid Goodhart problems "The important thing is to correctly model my uncertainty and overconfidence.", I can now refine that into:

  • The important thing is to correctly model my fuzziness, and my extrapolation desiderata.

Neat and elegant! However, to make it more applicable, I unfortunately need to extend it in a less elegant fashion:

  • The important thing is to correctly model my fuzziness, and my extrapolation desiderata, including any meta-desiderata I might have for how to model this correctly (and any errors I might be making, that I would desire to have recognised as errors).

Note that there is no longer any deep need to model "my" uncertainty. It is still important to model uncertainty about the real world correctly, and if I'm mistaken about the real world, this may be relevant to what I believe my extrapolation desiderata are. But modelling my uncertainty is merely instrumentally useful, but modelling my fuzziness is a terminal goal if we want to get it right.

As a minor example of the challenge of the above, consider that this would have needed to be able to detect that adversarial examples were problematic, before anyone had conceived of the idea.

I won't develop this too much more here, as the ideas will be included in my research agenda whose first draft should be published here soon.


Infinity is an adjective like positive rather than an amount

30 мая, 2019 - 16:22
Published on May 30, 2019 1:22 PM UTC

I often see the belief that infinity is a quantity, when I think it's a quality. This post introduces two qualitative notions on a field where people are likely to have a good quantitative understanding in order to show how the qualitative understanding isn't that satifying.

Assume natural numbers and then we will extend them with two new numbers. They will be suggestively named "inpositivity" and "positivity".

What these numbers are is characterised by rules involving them (I will not differentiate between axiom and theorem).

For any number a and b if a>b then b-a=inpositivity


inpositivity+a is undefined








positivity+inpositivity is undefined

positivity - inpositivity = positivity

Call the system defined as the "Dulled natural numbers". Now if you compare this system with integers you might see that the qualities "positive" and "negative" have a close correspondence with these two extra dulled numbers. And this is the intention of the construction. We could consider the dulled system in it's own right. Or we can see it as talking coarsely about the more detailed world of integers.

Note that for expressions explicitly listed as undefined we can find very easily reasons why they don't have locked-in results. For example 5-3=2 and 5-7=-2 their "coarsement" analgoues would be positivity-positivity=positivity and positivity-posititivy=inpositivity. Likewise a rule like "positivity-positivity=0" could take "detailment" forms like "5-7=0" which would be clearly false.

When we are talking about infinities one of the systems we might talk about is the extended real line. It's a similar kind of system in that we add two numbers to the usual real numbers. When there we have expressions like "+infinity - +infinity" being undefined we might not be equiped to have a "detailment" argument to cast it into another system why it is so. And after all if it is good system it must be fine at it's own merits.

But when people read statements involving those kind of infinite numbers they might be reading it as "THE infinity + THE infinity". But there is the option of reading it "AN infinity + AN infinity". In the same way you do not read "THE positivity + THE positivity" but "A positive number + A positive number". When you have expressed that a number is positive you have not yet said how big it is. Likewise when you say that an amount is infinite you have expressed a quality that limits some magnitudes to be out of question but you have not actually specified its magnitude.

As an aside: boolean multiplication and addition of 1*1=1 and 1+1=1 seems to be suspiciuosly similar to positivity multiplication and addition. It might be that boolean algebra is the dulled natural numbers without the positive numbers and inpositivity. Boolean 1 does not translate to natural number 1 and this would be true even if we didn't know about positivity. We could also form other "dulled" number systems such as dulled reals or dulled rationals. Making dulling into an operation that could be applied to arbitrary number systems could be interesting but I don't know the language in which a mathematician could read it as a stand-alone idea.


Quotes from Moral Mazes

30 мая, 2019 - 14:50
Published on May 30, 2019 11:50 AM UTC

Reading and actually paying attention to Moral Mazes is hard. Writing carefully about it is even harder. I effectively spent several months forcing my way through the book, because it seemed important to do that. I then spent a month trying to write about the book, but that’s going super slow as well. The repetition, the saying the same thing from multiple angles, the detailed examples, seem necessary to get the points across, because one has the very strong instinct to avoid understanding it, to read without seeing, to hear without listening. At least, I know I did, despite these things also not only not feeling new, but resonating with my direct experiences.

So in the interest of getting something out there, and hoping that I’ll be able to address things in more detail later, here are all the 168 (!) quotes I highlighted from the book, roughly organized into categories. Locations listed are how to find the quote in the Kindle edition, and the quotes are numbered for ease of search and reference.

A. Hierarchy and Credit

1. As a former vice-president of a large firm says: “What is right in the corporation is not what is right in a man’s home or in his church. What is right in the corporation is what the guy above you wants from you. That’s what morality is in the corporation.” (Location 148)

2. When managers describe their work to an outsider, they almost always first say: “I work for [Bill James]” or “I report to [Harry Mills]” or “I’m in [Joe Bell’s] group,”* and only then proceed to describe their actual work functions. (Location 387)

3. The key interlocking mechanism of this structure is its reporting system. Each manager gathers up the profit targets or other objectives of his or her subordinates and, with these, formulates his commitments to his boss; this boss takes these commitments and those of his other subordinates, and in turn makes a commitment to his boss.* (Location 441)

4. It is characteristic of this authority system that details are pushed down and credit is pulled up. (Location 446)

5. One gives credit, therefore, not necessarily where it is due, although one always invokes this old saw, but where prudence dictates. Customarily, people who had nothing to do with the success of a project can be allocated credit for their exemplary efforts. At the middle levels, therefore, credit for a particular idea or success is always a type of refracted social honor; one cannot claim credit even if it is earned. Credit has to be given, and acceptance of the gift implicitly involves a reaffirmation and strengthening of fealty. A superior may share some credit with subordinates in order to deepen fealty relationships and induce greater efforts on his behalf. Of course, a different system obtains in the allocation of blame. (Location 482)

6. If one has a mistake-prone boss, there is, of course, always the temptation to let him make a fool of himself, but the wise subordinate knows that this carries two dangers—he himself may get done in by his boss’s errors, and, perhaps more important, other managers will view with the gravest suspicion a subordinate who withholds crucial information from his boss even if they think the boss is a nincompoop. A subordinate must also not circumvent his boss nor ever give the appearance of doing so. He must never contradict his boss’s judgment in public. To violate the last admonition is thought to constitute a kind of death wish in business, and one who does so should practice what one executive calls “flexibility drills,” an exercise “where you put your head between your legs and kiss your ass good-bye.” (Location 424)

7. The general rule is that bosses are expected to protect those in their bailiwicks. Not to do so, or to be unable to do so, is taken as a sign of untrustworthiness or weakness. If, however, subordinates make mistakes that are thought to be dumb, or especially if they violate fealty obligations—for example, going around their boss—then abandonment of them to the vagaries of organizational forces is quite acceptable. (Location 438)

8. Managers often note that one must stay at least three drinks behind one’s boss at social functions; this meant that Brown’s subordinates might never drink at all on such occasions. (Location 630)

9. However, the belief of insiders in abstract goals is not a prerequisite for personal success; belief in and subordination to individuals who articulate organizational goals is. One must, however, to be successful in a bureaucratic work situation, be able to act, at a moment’s notice, as if official reality is the only reality. (Location 1194)

10. You can put the damper on anyone who works for you very easily and that’s why there’s too much chemistry in the corporation. There’s not enough objective information about people. When you really want to do somebody in, you just say, well, he can’t get along with people. That’s a big one. And we do that constantly. What that means, by the way, is that he pissed me off; he gave evidence of his frustration with some situation. Another big one is that he can’t manage—he doesn’t delegate or he doesn’t make his subordinates keep his commitments. So in this sort of way, a consensus does build up about a person and a guy can be dead and not even know it. (Location 1475)

11. Only at this point did Brady realize that it was the CEO himself who was fiddling with the numbers. The entire dual reporting system that the CEO had personally initiated was in part an elaborate spy network to guard against discovery of the slush fund manipulation, and perhaps other finagling, rather than a system to ensure financial honesty. (Location 2404) [Leads into next quote]

12. The corporate managers to whom I presented this case see Brady’s dilemma as devoid of moral or ethical content. In their view, the issues that Brady raises are, first of all, simply practical matters. His basic failing was, first, that he violated the fundamental rules of bureaucratic life. These are usually stated briefly as a series of admonitions. (1) You never go around your boss. (2) You tell your boss what he wants to hear, even when your boss claims that he wants dissenting views. (3) If your boss wants something dropped, you drop it. (4) You are sensitive to your boss’s wishes so that you anticipate what he wants; you don’t force him, in other words, to act as boss. (5) Your job is not to report something that your boss does not want reported, but rather to cover it up. You do what your job requires, and you keep your mouth shut. Second, the managers that I interviewed feel that Brady had plenty of available legitimations to excuse or justify his not acting. Clearly, they feel, a great many other executives knew about the pension fund scam and did nothing; everybody, especially the top bosses, was playing the game. The problem fell into other people’s areas, was their responsibility, and therefore their problem. Why, then, worry about it? Besides, Brady had a number of ways out of the situation if he found it intolerable, including resigning. Moreover, whatever action he took would be insignificant anyway so why bother to act at all and jeopardize himself? Even a fool should have known that the CEO was not likely to take whatever blame resulted from the whole affair. (Location 2429)

13. One’s best efforts at being fair, equitable, and generous with subordinates clash both with a logic that demands choices between people, inevitably producing hatred, envy and animosity, and with the plain fact that, despite protestations to the contrary, many people do not want to be treated fairly. (Location 4552)

14. Mastering the subtle but necessary arts of deference without seeming to be deferential, of “brown nosing” without fawning, of simultaneous self-promotion and self-effacement, and occasionally of the outright self-abasement that such relationships require is a taxing endeavor that demands continual compromises with conventional and popular notions of integrity. Only those with an inexhaustible capacity for self-rationalization, fueled by boundless ambition, can escape the discomfort such compromises produce. (Location 4572)

B. Feeling Comfortable

15. Essentially, managers try to gauge whether they feel “comfortable” with proposed resolutions to specific problems, a task that always involves an assessment of others’ organizational morality and a reckoning of the practical organizational and market exigencies at hand. The notion of comfort has many meanings. When applied to other persons, the idea of comfort is an intuitive measure of trustworthiness, reliability, and predictability in a polycentric world that managers often find troubling, ambiguous, and anxiety-laden. Such assessment of others’ organizational morality is a crucial aspect of a more general set of probations that are intrinsic to managerial work. (Location 302)

16. They objected in particular to those aspects of my brief written proposal that discussed the ethical dilemmas of managerial work. They urged me to avoid any mention of ethics or values altogether and concentrate instead on the “decision-making process” where I could talk about “trade-offs” and focus on the “hard decisions between competing interests” that mark managerial work. Taking these cues, I rewrote and rewrote the proposal couching my problem in the bland, euphemistic language that I was rapidly learning is the lingua franca of the corporate world. But such recasting eroded whatever was distinctive about the project and some managers dismissed the study as a reinvention of the wheel. (Location 324)

17. In effect, I could not get access to study managers’ moral rules-in-use because I seemed unable to articulate the appropriate stance that would convince key managers that I already understood those rules and was thus a person with whom they could “feel comfortable” enough to trust. (Location 334)

18. The process centered on the written proposal that I had been circulating and consisted essentially of a furthering of my linguistic education in the art of indirect rather than pointed statement and, more particularly, a reformulation of my inquiry that recast the moral issues of managerial work as issues of public relations. When, after several rewritings, the proposal satisfied him, he approached a well-placed executive in a large textile firm that I have given the pseudonym of Weft Corporation and vouched for me. At that point, the proposal itself became meaningless since, to my knowledge, no one except the two executives who arranged access ever saw it. The personal vouching, however, was crucial. This was based on what both men took to be a demonstrated willingness and ability to be “flexible” and especially on their perception that I already grasped the most salient aspect of managerial morality as managers themselves see it—that is, how their values and ethics appear in the public eye. (Location 347)

19. At bottom, all of the social contexts of the managerial world seek to discover if one “can feel comfortable” with another manager, if he is someone who “can be trusted,” if he is “our kind of guy,” or, in short, if he is “one of the gang.” (Location 905)

20. My search for access involved me in some of the crucial bureaucratic intricacies that shape managers’ experiences. These include organizational upheavals, political rivalries, linguistic ambiguity, the supremacy of chance and tangled personal connections over any notion of intrinsic merit, the central significance of public relations, and, perhaps especially, the ceaseless moral probations for inclusion in a managerial circle. Managers keep their eyes on the organizational premiums that shape behavior, values, ethics, and worldviews in corporate bureaucracies. I focus on those premiums… (Location 387)

21. One becomes known, for instance, as a trusted friend of a friend; thought of as a person to whom one can safely refer a thorny problem; considered a “sensible” or “reasonable” or, especially, a “flexible” person, not a “renegade” or a “loose cannon rolling around the lawn”; known to be a discreet person attuned to the nuances of corporate etiquette, one who can keep one’s mouth shut or who can look away and pretend to notice nothing; or considered a person with sharp ideas that break deadlocks but who does not object to the ideas being appropriated by superiors. (Location 870)

22. Similarly, Covenant’s CEO sold large tracts of land with valuable minerals at dumbfoundingly low prices. The CEO and his aides said that Covenant simply did not have the experience to mine these minerals efficiently, a self-evident fact from the low profit rate of the business. In all likelihood, according to a manager close to the situation, the CEO, a man with a financial bent and a ready eye for the quick paper deal, felt so uncomfortable with the exigencies of mining these minerals that he ignored the fact that the prices the corporation was getting for the minerals had been negotiated forty years earlier. Such impulsiveness and indeed, one might say from a certain perspective, irrationality, is, of course, always justified in rational and reasonable terms. It is so commonplace in the corporate world that many managers expect whatever ordered processes they do erect to be subverted or overturned by executive fiat, masquerading as an established bureaucratic procedure or considered judgment. (Location 1700)

23. think that I’ve got to where I am today because of this. [His boss’s boss] knows that I saved the company a lot of money and a lot of asses to boot. And he and others know that I am someone who can be trusted. I can keep my mouth shut…. And that’s the biggest thing that I have going for me—that people feel that I can be trusted. I can’t overemphasize that enough. (Location 2917)

24. Only those men and women who allow peers and superiors to feel morally comfortable in the ambiguous muddles of the world of affairs have a chance to survive and flourish in big organizations when power and authority shift due to changes in markets, internal power struggles, or the need to respond to external exigencies. (Location 4943)

C. Struggle for Success

25. The logical result of alertness to expediency is the elimination of any ethical lines at all. (Location 2985)

26. The two areas are, of course, related since one’s chances in an organization depend largely on one’s “credibility,” that is, on the widespread belief that one can act effectively. One must therefore prevail regularly, though not always, in small things to have any hope of positioning oneself for big issues. The hidden agenda of seemingly petty disputes may be a struggle over long-term organizational fates. (Location 812)

27. A fundamental rule of corporate politics is that one never cedes control over assets, even if the assets are administrative headaches. (Location 643)

28. Bureaucratic hierarchies, simply by offering ascertainable rewards for certain behavior, fuel the ambition of those men and women ready to subject themselves to the discipline of external exigencies and of their organization’s institutional logic, the socially constructed, shared understanding of how their world works. However, since rewards are always scarce, bureaucracies necessarily pit people against each other and inevitably thwart the ambitions of some. (Location 806)

29. When asked who gets ahead, an executive vice-president at Weft Corporation says: The guys who want it [get ahead]. The guys who work. You can spot it in the first six months. They work hard, they come to work earlier, they leave later. They have suggestions at meetings. They come into a business and the business picks right up. They don’t go on coffee breaks down here [in the basement]. You see the parade of people going back and forth down here? There’s no reason for that. I never did that. If you need coffee, you can have it at your desk. Some people put in time and some people work. (Location 992)

30. Proper management of one’s external appearances simply signals to one’s peers and to one’s superiors that one is prepared to undertake other kinds of self-adaptation. Managers also stress the need to exercise iron self-control and to have the ability to mask all emotion and intention behind bland, smiling, and agreeable public faces. (Location 1059)

31. The price of bureaucratic power is a relentlessly methodical subjection of one’s impulses, at least in public. (Location 1100)

32. [Style] is being able to talk easily and make presentations. To become credible easily and quickly. You can advance quickly even without technical experience if you have style. You get a lot of points for style. You’ve got to be able to articulate problems, plans, and strategies without seeming to have to refer to all sorts of memos and so on. The key in public performances and presentations is in knowing how to talk forcefully without referring to notes and memoranda. To be able to map out plans quickly and surely. (Location 1298)

33. As one manager says: “Personality spells success or failure, not what you do on the field.” (Location 1383)

34. More generally, there are several rules that apply here. First, no one in a line position—that is, with responsibility for profit and loss—who regularly “misses his numbers” will survive, let alone rise. Second, a person who always hits his numbers but who lacks some or all of the required social skills will not rise. Third, a person who sometimes misses his numbers but who has all the desirable social traits will rise. (Location 1412)

35. A managerial commonplace says: “In the corporate world, 1,000 ‘Attaboys’ are wiped away with one ‘Oh, shit!’” (Location 1619)

36. There’s a lot of it [fear and anxiety]. To a large degree it’s because people are more honest with themselves than you might believe. People know their own shortcomings. They know when they’re over their heads. A lot of people are sitting in jobs that they know are bigger than they should be in. But they can’t admit that in public and, at still another level, to themselves. The organizational push for advancement produces many people who get in over their heads and don’t know what they are doing. And they are very fearful of making a mistake and this leads to all sorts of personal disloyalty. But people know their capabilities and know that they are on thin ice. And they know that if they make mistakes, it will cost them dearly. So there’s no honesty in our daily interaction and there’s doubt about our abilities. The two go together. (Location 1767)

37. One comes to gauge that hard-won access to managerial circles takes precedence over fussing with abstract principles. (Location 2923)

38. Perceptions of pervasive mediocrity breed an endless quest for social distinctions even of a minor sort that might give one an “edge,” enable one to “step out of the crowd,” or at least serve as a basis for individual claims to privilege. More specifically, an atmosphere of mediocrity erodes the hope of meaningful collective achievement and encourages, at least among more aggressive managers, a predatory stance toward their organizations, that is, a search for private deals, a working of the system for one’s own personal advantage. (Location 4436)

39. One leaves behind as well the technical knowledge or scientific expertise of one’s younger years, lore now more suited for the narrower roles of technicians or junior managers. One must, in fact, put distance between oneself and technical details of every sort or risk the inevitable entrapment of the particular. Salesmen, too, must leave their bags and regular customers and long boisterous evenings that seal measurable deals behind them and turn to marketing strategies. Work becomes more ambiguous, directed as it is toward maneuvering money, symbols, organizational structures, and especially people. The CEO at Weft Corporation, it is said, “doesn’t know a loom from a car.” And the higher one goes, the more managers find that “the essence of managerial work is cronyism, covering your ass, [and] pyramiding to protect your buddies.” (Location 4539)

40. the rewards of corporate success can be very great. And those who do succeed, those who find their way out of the crowded, twisting corridors and into the back rooms where the real action is, where the big games take place, and where everyone present is a player, shape, in a decisive way, the moral rules-in-use that filter down through their organizations. The ethos that they fashion turns principles into guidelines, ethics into etiquette, values into tastes, personal responsibility into an adroitness at public relations, and notions of truth into credibility. (Location 4603)

41. Instead, success becomes contingent on others’ interpretations of one’s performance, leading to a break in the accepted moral economy between talent combined with effort and reward. This makes compulsive sociability an occupational virtue, as one attempts to discern and shape peers’ and superiors’ interpretations. (Location 4950)

D. Nothing Matters, But Hit Your Numbers

42. Managers rarely speak of objective criteria for achieving success because once certain crucial points in one’s career are passed, success and failure seem to have little to do with one’s accomplishments. (Location 917)

43. Corporations rely on other institutions—principally the schools—to establish what might be called competence hurdles. The demonstrated ability of a student to leap over successively higher hurdles in school is taken as evidence of the ability to weather well the probationary trials of corporate life. (Location 920)

44. In Alchemy Inc., whether in sales, marketing, manufacturing, or finance, the “breaking point” in the hierarchy is generally thought to be grade 13 out of 25 or the top 8.5 percent of management. By the time managers reach such a numbered grade in an ordered hierarchy—and the grade is socially defined and varies from company to company—managerial competence as such is taken for granted and assumed not to differ greatly from one manager to the next. (Location 943)

45. A product manager in the chemical company talks about the lack of connection between work and results: I guess the most anxiety provoking thing about working in business is that you are judged on results whether those results are your fault or not. So you can get a guy who has tried really hard but disaster strikes; and you can get a guy who does nothing and his business makes a big success. And so you just never know which way things are going to go and you’re never sure about the relationship of your work to the outcome. One of the top executives in Weft Corporation echoes this sentiment: I always say that there is no such thing as a marketing genius; there are only great markets. (Location 1585)

46. Assuming a basic level of corporate resources and managerial know-how, real economic outcome is seen to depend on factors largely beyond organizational or personal control. (Location 1592)

47. upper-middle level manager says: If I were just out of school and somebody told me that it doesn’t matter what you do and how well you do it but that what matters is being in the right place at the right time, I’d have said that hard work is still the key. You know, the old virtues. But now as I have gotten older, I think it’s pure happenstance—luck. Things happen to people and being in the right time and place and knowing the right people is the key. (Location 1621)

48. It is interesting to note in this context that a line manager’s long-run credibility suffers just as much from missing his numbers on the up side (that is, achieving profits higher than predicted) as from missing them on the down side although, as one might expect, the immediate consequences of such different miscalculations vary. Both outcomes, however, undercut the ideology of managerial planning and control. (Location 1595)

49. A top staff official at Covenant Corporation explains: By putting the money in this business, you’re taking the money away from others. In human terms, that’s what you’re doing. It’s money that you could provide jobs with to others. So when you get a guy in the business who comes in under or over the plan, well, both are equally suspect. Because you’re making major decisions based on your plan…. Like when we shut down [business A] and put the money into [business B], the whole legitimacy of the operation depends on the [business A] guys accepting the rationale that more money can be made in another operation. (Location 1599)

50. of a plant manager who, when his machinery had ground to a halt and his technicians were baffled and everyone turned to him to make a decision, told his crew, without the faintest idea of the right thing to do and with the great fear that all he had worked for was about to crumble before him, to dump ten pounds of phosphate into the machine. The machine sprang to life and he became a hero. (Location 1656)

E. Implicitness

51. If I tell someone what to do—like do A, B, or C—the inference and implication is that he will succeed in accomplishing the objective. Now, if he doesn’t succeed, that means that I have invested part of myself in his work and I lose any right I have to chew his ass out if he doesn’t succeed. If I tell you what to do, I can’t bawl you out if things don’t work. And this is why a lot of bosses don’t give explicit directions. They just give a statement of objectives, and then they can criticize subordinates who fail to make their goals. (Location 454)

52. A typical example occurred in Weft Corporation a few years ago when the CEO, new at the time, expressed mild concern about the rising operating costs of the company’s fleet of rented cars. The following day, a stringent system for monitoring mileage replaced the previous casual practice. (Location 490)

53. One must remember, for instance, that in our litigious age the best rule in dealing with angry subordinates is to say nothing or as little as possible since whatever one says may be used against oneself and one’s organization. (Location 1066)

54. Well, usually you don’t tell people the truth. I once knew a guy whom I knew was about to be fired and I asked if he had been told and he had never been told. I think you should tell people explicitly. Things like that shouldn’t have to be decoded. But you can understand how it happens. Suppose you have a guy and the consensus is that he isn’t promotable. You wouldn’t ever—or very seldom—tell him. He goes on to justify his silence: There are people who go through life thinking they can do a lot more than they really can do. And the reason is that losing or changing jobs is a very high stress situation and most people prefer to hang on to what they’ve got—to their routine. They’re not happy but they go through life like prisoners of war not recognizing their true situation. (Location 1494)

55. You get the situation where a lot of people don’t really want to know…. Like one guy we have, he will retire on his job. He’s in my division. He knows it. I know it. And he doesn’t want me to tell him about that. Now don’t ask me how I know that but, believe me, I do. (Location 1502)

56. Why does it happen? Because people are afraid of confrontations. People want to be thought of as kind, sensitive, and compassionate. Being compassionate has a good significance in our society. The easy way out is not to do anything, don’t tell the guy. That happens a lot. (Location 1510)

57. As a matter of fact, he wouldn’t even have to say “cut capital.” He would just put pressure on him by saying: “Well, sales are down 50 percent; why aren’t your expenses down 50 percent?” My boss will come to me, by the time it reaches him, and say: “Cut costs.” It’s as simple as that. (Location 2078)

58. He put things into writing in a world that, apart from ritual nods to the importance of documentation, actually fosters ambiguity by its reliance on talk as the basic mode of negotiation and command. Talk, of course, lends itself more readily than documents to backtracking, filling in, evasion, subterfuge, and secrecy, all important virtues if one is to do what has to be done while establishing and maintaining the kinds of relationships that alone can protect oneself. (Location 2636)

59. It is unlikely, however, that any workers affected could ever piece things together. First, there is nothing in writing. Second, Tucker feels sure that everyone involved would, if it became necessary, simply deny knowledge and claim that the process was altered solely for production reasons. 4. Finally, he says: The basic rule is that you hope that these kinds of things never occur. Nobody wants to hurt people. Nobody would ever consciously plan to do something that would endanger people. But when things happen, well, you cover for yourself and your company. (Location 2948)

60. Discreet suggestions, hints, and coded messages take the place of command; this, of course, places a premium on subordinates’ abilities to read correctly their bosses’ vaguely articulated or completely unstated wishes. One cannot even criticize one’s subordinates to one’s own superior without risking a negative evaluation of one’s own managerial judgment. (Location 3009)

61 (Chart). Phrase / Probable Intended Meaning

*Exceptionally well qualified  / Has committed no major blunders to date

Tactful in dealing with superiors / Knows when to keep his mouth shut

Quick thinking / Offers plausible excuses

Meticulous attention to detail / A nitpicker

Slightly below average / Stupid

Unusually loyal / Wanted by no one else

Indifferent to instruction / Knows more than one’s superior

Strong adherence to principles / Stubborn

Requires work-value attitudinal readjustment / Lazy and hardheaded (Location 3018)

62. For the most part, euphemistic language is not used with the intent to deceive. Managers past a certain point, as suggested earlier, are assumed to be “maze-bright” and able to “read between the lines” of a conversation or a memorandum and to distinguish accurately suggestions from directives, inquiries from investigations, and bluffs from threats. Managers who are “maze-dense,” like the manager at Weft Corporation who, though told somewhat indirectly that he was fired, did not realize his fate until the following day, might consider the oblique, elliptical quality of managerial language to skirt deceit. However, most often when managers use euphemistic language with each other (and it is important to remember that in private among trusted others their language can be very direct, colorful, and indeed earthy), its principal purpose is to communicate certain meanings within specific contexts with the implicit understanding that should the context change, a new, more appropriate meaning can be attached to the language already used. In this sense, the corporation is a place where people are not held to what they say because it is generally understood that their word is always provisional. (Location 3034)

63. The rule of thumb here seems to be that the more troublesome a problem, the more desiccated and vague the public language describing it should be. Of course, when a troublesome problem bursts into public controversy, euphemism becomes a crucial tool of those managers who have to face the public in some forum. The task here is to defuse public criticism and sometimes outrage with abstract unemotional characterizations of issues. (Location 3044)

F. Short Term Thinking

64. The ideal situation, of course, is to end up in a position where one can fire one’s successors for one’s own previous mistakes. (Location 2102)

65. Moreover, the only real threat to corporations on environmental issues was in the courts, which, however, judge past actions, not present practices. By the time the courts get to cases generated by contemporary practices, typically in fifteen years, those executives presently in charge will have moved on, leaving any problems their policies might create to others. (Location 707)

66. These choices also reflect judgments about whether a corporation can offer the hard-charging MBAs from the top-ranked schools enough quick variety to retain them long enough to justify their inflated salaries. (Location 926)

67. Similarly, accounting systems that place a premium on bare-bones inventory reflect the same pressure for short-run profit maximization. For instance, at Covenant Corporation the story is told about a plant that produced a useful by-product at no extra cost. One simply had to store it until it was needed for other internal operations. Covenant, however, works with an accounting system that considers by-products as inventory; moreover, inventory counts against one at the end of a fiscal year. In order to cut costs, managers decided to throw out the by-product at the end of a financial cycle. But a sudden shortage of the material trebled its cost two months later. To service their own operations, managers had to go hat in hand to their competitors to buy the material at the premium prices. (Location 1837)

68. This sets the stage for financial sharpshooters who, in takeover strategies, buy large chunks of a company’s stock at devalued prices only to be “greenmailed” (persuaded with financial inducements) by the target company’s management into surrendering these blocks of holdings at premium prices. In such unsettled times, where virtually any large corporation could become a takeover target, managers feel that they have to keep their companies’ stock properly valued. As it happens, the markets honor only short-term gains. (Location 1854)

69. Instead, one focuses attention on important problems of the moment that must be solved. Since these are always plentiful, they justify postponing less pressing concerns. Of course, managers know at one level of their consciousness that today’s minor issues can quickly become tomorrow’s major crises, but the pressure for annual, quarterly, monthly, daily, and even hourly “results,” that is, measurable progress plausibly attributed to one’s own efforts, crowds out reflection about the future. An upper-middle manager at Alchemy Inc. recalls, for instance, his days as a plant manager when his boss at company headquarters phoned him every three hours to see how many tons of soda ash had been produced in the interval. (Location 1869)

70. This goes to the heart of the problem. Managers think in the short run because they are evaluated by both their superiors and peers on their short-term results. Those who are not seen to be producing requisite short-run gains come to be thought of as embarrassing liabilities. Of course, past work gets downgraded in such a process. The old saw, still heard frequently today, “I know what you did for me yesterday, but what have you done for me lately?” is more than a tired garment district salesman’s joke. It accurately reflects the widespread amnesia among managers about others’ past accomplishments, however notable, and points to the probationary crucibles at the core of managerial life. Managers feel that if they do not survive the short run, the long run hardly matters, and one can only buy time for the future by attending to short-term goals. As one manager says: “Our horizon is today’s lunch.” (Location 1875)

71. Now you see this at work with mistakes. You can make mistakes in the work you do and not suffer any consequences. For instance, I could negotiate a contract that might have a phrase that would trigger considerable harm to the company in the event of the occurrence of some set of circumstances. The chances are that no one would ever know. But if something did happen and the company got into trouble, and I had moved on from that job to another, it would never be traced to me. The problem would be that of the guy who presently has responsibility. And it would be his headache. There’s no tracking system in the corporation. Some managers argue that outrunning mistakes is the real meaning of “being on the fast track,” the real key to managerial success. The same lawyer continues: In fact, one way of looking at success patterns in the corporation is that the people who are in high positions have never been in one place long enough for their problems to catch up with them. They outrun their mistakes. That’s why to be successful in a business organization, you have to move quickly. (Location 2013)

72. Both Covenant Corporation and Weft Corporation, for instance, place a great premium on a division’s or a subsidiary’s return on assets (ROA); managers who can successfully squeeze assets are first in line, for instance, for the handsome rewards allotted through bonus programs. One good way for business managers to increase their ROA is to reduce assets while maintaining sales. Usually, managers will do everything they can to hold down expenditures in order to decrease the asset base at the end of a quarter or especially at the end of the fiscal year. The most common way of doing this is by deferring capital expenditures, everything from maintenance to innovative investments, as long as possible. Done over a short period, this is called “starving a plant”; done over a longer period, it is called “milking a plant.” (Location 2034)

73. A plant that is not well maintained will fail in the short term, so you have to spend money there; a plant that has poorly trained people will fail today, so you have to spend money there. But you can still milk (Location 2045)

74. We’re judged on the short-term because everybody changes their jobs so frequently. (Location 2042)

75. My favorite things are not to replace my stores inventory and that shows up as direct profit on your balance sheet; not replace people who retire, and stretch everybody else out; cut down on overtime; cut working inventories to the bone. [You can also] lower the quality standards; you can get away with this in the short term because people will accept that for awhile, though in the long term people will stop buying from you. (Location 2050)

76. At the very top of organizations, one does not so much continue to outrun mistakes as tough them out with sheer brazenness. In such ways, bureaucracies may be thought of, in C. Wright Mills’s phrase, as vast systems of organized irresponsibility. (Location 2123)

77. When a weaver is unable to repair a stop, she shuts down the loom and flags a “fixer,” the most skilled and highest-ranking worker on the shop floor, who does the repair or basic maintenance necessary to get the loom working again. However, since weavers are paid by the piece and can make no gain from a loom out of service for repair or maintenance during their own shifts, weavers will tend to remedy stops in any way they can in order to keep their cloth production high, leaving the maintenance and repair of the machinery and the economic cost involved to another weaver on another shift. Supervisors and managers who are evaluated by a plant’s overall weaving efficiency are thus forced to monitor the number of loom stops constantly in order to make sure that looms badly in need of repair or maintenance get proper attention. Whenever structural inducements place premiums on immediate personal gains, especially when mistakes are not penalized, there seems to be a sharp decline in the likelihood of men and women sacrificing their own interests for others, for their organizations, or least of all for the common weal.2 (Location 2135)

78. It was said that Noll had milked and milked thoroughly every plant he ever supervised. One day, a story goes, he was accused of this in a public meeting by a vice-president who was then his superior. Noll is said to have responded with great boldness: “[Joe], how can you sit there and say that to me? How in the hell do you think you got to where you are and how do you think you stay there?” (Location 2153)

79. If a guy keeps moving, he can say, “Look, I ran this plant better than my predecessors.” And people have to concede that. A lot of people do that. Then you get the guy who takes his place and tries to run things right and he has to spend a lot of money. And people look at the guy who was there before and they say: “Well, old [Noll] ran the plant well and he didn’t have to spend any money like you’re claiming you do.” I don’t think there is anything wrong with milking a plant. As long as you know you’re milking (Location 2223)

80. No, definitely not. Would any sane, rational man spend $15 million for a 2 percent return? … Now it does improve the dust levels, but it was that if we don’t invest the money now, we would be in a desperate [competitive] position fifteen years from now. Our demonstrated cash flow situation was such that eventually we would have had even tougher decisions to make. (Location 3579)

81. Executives also admit, somewhat ruefully and only when their office doors are closed, that OSHA’s regulation on cotton dust has been the main factor in forcing the pace of technological innovation in a centuries-old, hidebound, and somewhat stagnant industry. It has also been a major factor in forcing executives to think in the long run rather than continually succumbing to short-term pressures. This is one of the reasons why the shrewdest among them only feigned elation at the attempts by Reagan’s OSHA appointees to remove the cotton dust regulation from the purview of the Supreme Court. If such a move were successful, it could only encourage the traditionally reactionary elements of the textile industry who refuse to recognize on principle that government regulation, within reason, can be the businessman’s best friend. (Location 3593)

82. S&Ls could now open transaction accounts and make commercial real estate loans, regardless of geography, up to 40 percent of their assets. This prompted many S&L executives to sell their long-term, fixed-rate, low- or no-profit mortgages to Wall Street firms for as little as 60 percent of their value in order to obtain ready funds for much more lucrative, though much riskier, investments. (Location 4649)

83. Organized irresponsibility has migrated to the nooks and crannies of our society. One sees presidents of universities and colleges who preside over the destruction of their institutions’ endowments, not to mention the intellectual integrity of curricula, and then outrun their mistakes and move on to higher academic and even top political posts. One sees intellectuals who are completely committed to the destruction of the Western tradition that nurtures them and who help create the profound social and cultural centrifugality that marks American society. One sees big-city mayors who cede control of the streets to drug dealers and other thugs for fear of being labeled nonprogressive by various advocacy groups and the press. One sees long-time political insiders in Washington who “screw up and move up,” an aphorism that aptly captures the ethos of government bureaucracies. And doublethink and doublespeak now permeate public discourse, bewildering even intelligent, thoughtful citizens. (Location 4990)

G. Social Politics

84. Managers’ cryptic aphorism, “Well, you never know…,” repeated often and regularly, captures the sense of uncertainty created by the constant potential for social reversal. Managers know too, and take for granted, that the personnel changes brought about by upheavals are to a great extent arbitrary and depend more than anything else on one’s social relationships with key individuals and with groups of managers. (Location 761)

85. “Every big organization is set up for the benefit of those who control it; the boss gets what he wants.” (Location 827)

86. In any event, just as managers must continually please their boss, their boss’s boss, their patrons, their president, and their CEO, so must they prove themselves again and again to each other. Work becomes an endless round of what might be called probationary crucibles. Together with the uncertainty and sense of contingency that mark managerial work, this constant state of probation produces a profound anxiety in managers, perhaps the key experience of managerial work. (Location 910)

87. See, once you are at a certain level of experience, the difference between a vice-president, an executive vice-president, and a general manager is negligible. It has relatively little to do with ability as such. People are all good at that level. They wouldn’t be there without that ability. So it has little to do with ability or with business experience and so on. All have similar levels of ability, drive, competence, and so on. What happens is that people perceive in others what they like—operating styles, lifestyles, personalities, ability to get along. Now these are all very subjective judgments. And what happens is that if a person in authority sees someone else’s guy as less competent than his own guy, well, he’ll always perceive him that way. And he’ll always pick—as a result—his own guy when the chance to do so comes up. (Location 1013) [Also see Nothing Matters]

88. Striking, distinctive characteristics of any sort, in fact, are dangerous in the corporate world. One of the most damaging things, for instance, that can be said about a manager is that he is brilliant. This almost invariably signals a judgment that the person has publicly asserted his intelligence and is perceived as a threat to others. What good is a wizard who makes his colleagues and his customers uncomfortable? (Location 1173)

89. Equally damaging is the judgment that a person cannot get along with others—he is “too pushy,” that is, he exhibits too much “persistence in getting to the right answers,” is “always asking why,” and does not know “when to back off.” Or he is “too abrasive,” or “too opinionated,” unable “to bend with the group.” Or he is a “wildman” or a “maverick,” that is, someone who is “outspoken.” Or he may be too aloof, too distant, “too professional.” (Location 1176)

90. The knowledgeable practitioners of corporate politics, whether patrons or leaders of cliques and networks, value nothing more highly than at least the appearance of unanimity of opinion among their clients and allies, especially during times of turmoil. (Location 1197)

91. Managers know that patrons and powerful allies protect those already selected as rising stars from the negative judgments of others; and only the foolhardy point out even egregious errors of those in power or those destined for it. (Location 1463)

92. Managers know that to be weak in a world that extols strength and power is to invite abuse. (Location 1536)

93. Such social distancing has two purposes: it undermines in advance or lays the groundwork for refusal of any claims that a person considered a failure might make on another, and it forestalls the possibility of being linked with that person in others’ cognitive maps. This becomes particularly important when there has been a known past association between oneself and one thought to have failed in some way. (Location 1542)

94. In fact, when it is even suspected that a person might be headed for trouble, anticipatory avoidance is the rule. (Location 1550)

95. Being a “fall guy,” that is, “taking the rap” or “taking the heat” for others’ decisions or mistakes is probably the most common kind of blame in big organizations. (Location 1904)

96. You have to remember that you only get to explain things away once. When things get screwed up, you get one chance. That’s why it’s important for everybody to be in bed with everybody else. And if they don’t like you from the start, you don’t have a chance. Because when things go wrong, what people do is sit down and say—without saying it in so many words—look, our jobs are on the line. Let’s make sure that it’s not us who gets nailed. (Location 1999)

97. Wilson and his Site Operations staff were particularly concerned because when the polar crane was finally used, it would be Site Operations who used it. They knew that, if they were in charge when the button was pushed, they could be blamed for whatever might go wrong. If the polar crane failed, it would be seen as the fault of Site Operations. (Location 2557)

98. high-ranking staff official at Covenant explains: Anxiety is endemic to anyone who works in a corporation. By the time you get to be middle management, it’s difficult to make friends because the normal requirement for friendship—that is, loyalty—doesn’t fit in this context. You have to look out for number one more than anything else. Moreover, the prevailing view is that managers are big boys and girls, well-paid, and should be able to take care of themselves. Besides, one person’s failure represents another person’s opportunity. (Location 1554)

99. of a young manager who, while at a company conference, went out for his weight-controlling 5:30 A.M. jog only to meet a vice-president similarly engaged, a powerful executive who now cheers the younger man’s work and presentations and introduces him to other influential senior managers; (Location 1654)

100. scattering ducks already set in a row. Besides, one can only criticize something when one has the resources to solve it in a clear and decisive way. Otherwise, one should keep one’s skepticism to oneself and get “on board.” (Location 2629)

101. Other managers and managerial cliques are always on the lookout for others’ mistakes or for actions that can be construed as mistakes and will pounce on anyone foolish enough to admit them. Even if others restrain an immediate attack, the knowledge of someone’s mistakes is ammunition for the future. Many managers “lay in the weeds, with rocks, and wait.” One who exposes a colleague’s errors in such a context and makes him vulnerable to others evinces, of course, only a fundamental untrustworthiness, unless one’s colleague has first betrayed oneself or others in some way— (Location 2858)

102. There is a premium in the higher circles of management on seeming fresh, dynamic, innovative, and up-to-date. In their social minglings and shoptalk with one another, particularly with their opposite numbers in other large companies, say, at the Business Roundtable, at high-level conferences at prestigious business schools, at summer galas in the Hamptons, or at the Super Bowl, the biggest business extravaganza of all, executives need to seem abreast of the latest trends in managerial know-how. No one wants to appear stodgy before one’s peers nor to have one’s firm defined in managerial networks, and perhaps thence to Wall Street, as “slow on the uptake.” Executives trade ideas and schemes and judge the efficacy of consultant programs not by any detached critical standards but by what is socially acceptable, desirable, and, perhaps most important, current in their circles. (Location 3155)

103. So they attend the sessions and with a seemingly dutiful eagerness learn literally to repeat the requisite formulas under the watchful eyes of senior managers. Senior managers do not themselves necessarily believe in such programs. In one seminar that I attended, the senior manager in charge startled a room of juniors by saying: Fellows, why aren’t any of you asking about the total lack of correspondence between what we’re preaching here and the way we run our company? But such outspokenness is rare. (Location 3209)

104. They admit, however, that the marvelously high fees that consultants command (in 1986 as high as $2,000 a day in New York City) enhance their legitimacy and encourage managers to lend credence to their schemes. (Location 3216)

105. A choice between securing one’s own success by jumping on and off the bandwagon of the moment, or sacrificing oneself for the long-run good of a corporation by diverting resources and really seeing a program through is, for most managers, no choice at all. Ambitious managers see self-sacrificing loyalty to a company as foolhardy. Moreover, middle and upper-middle level managers upon whom requests for self-sacrifice for the good of the organization are most likely to fall do not see top executives sacrificing themselves for the common good. For example, just after the CEO of Covenant Corporation announced one of his many purges, legitimated by a “comprehensive assessment of the hard choices facing us” by a major consulting firm, he purchased a new Sabre jet for executives and a new 31-foot company limousine for his own use at $1,000 a foot. He then flew the entire board of directors to Europe on the Concorde for a regular meeting to review, it was said, his most recent cost-cutting strategies. As other managers see it, bureaucratic hierarchy gives top bosses the license to act in their own interests and to pursue with impunity the arts of contradiction. (Location 3220)

H. The I in Team

106. The main dimensions of team play are as follows. 1. One must appear to be interchangeable with other managers near one’s level. Corporations discourage narrow specialization more strongly as one goes higher. They also discourage the expression of moral or political qualms. One might object, for example, to working with chemicals used in nuclear power, or working on weapons systems, and most corporations today would honor such objections. Publicly stating them, however, would end any realistic aspirations for higher posts because one’s usefulness to the organization depends on versatility. As one manager in Alchemy Inc. comments: “Well, we’d go along with his request but we’d always wonder about the guy. And in the back of our minds, we’d be thinking that he’ll soon object to working in the soda ash division because he doesn’t like glass.” Strong convictions of any sort are suspect. One manager says: If you meet a guy who hates red-haired persons, well, you’re going to wonder about whether that person has other weird perceptions as well. You’ve got to have a degree of interchangeability in business. To me, a person can have any beliefs they want, as long as they leave them at home. Similarly, one’s spouse’s public viewpoints or activities could reduce others’ perceptions of a manager’s versatility or indeed ability. In reference to another manager whose wife was known to be active in environmental action groups, lobbying in fact for legislation on chemical waste disposal, one Alchemy manager says: “If a guy can’t even manage his own wife, how can he be expected to manage other people?” (Location 1137)

107. Team play also means, as one manager in the chemical company puts it, “aligning oneself with the dominant ideology of the moment” or, as another says, “bowing to whichever god currently holds sway.” Such ideologies or gods may be thought of as official definitions of reality. (Location 1188)

108. You can indict a person by saying that he’s not a team player. That doesn’t mean he won’t follow directions. It’s because he voices an objection, because he argues with you before doing something, especially if he’s right. That’s when we really get mad—when the other guy is right. If he’s wrong, we can be condescending and adopt the “you poor stupid bastard” tone…. (Location 1220)

109. A team player is a manager who does not “force his boss to go to the whip,” but, rather, amiably chooses the direction his boss points out. Managers who choose otherwise or who evince stubbornness are said to “have made a decision,” a phrase almost always used to describe a choice that will shorten a career. (Location 1229)

110. Team players display a happy, upbeat, can-do approach to their work and to the organization. (Location 1252)

I. Avoiding Decision Making

111. You know that old saying: “Success has many parents; failure is an orphan”? Well, that describes decision making. A lot of people don’t want to make a commitment, at least publicly. This is a widespread problem. They can’t make judgments. They stand around and wait for everybody else’s reaction. Let me tell you a story which perfectly illustrates this. There was a [museum] collection coming, the [Arctic] collection, and there was a great deal of interest among designers in [Arctic] things. My own feeling was that it wouldn’t sell but I also recognized that everybody wanted to do it. But in this case, [our] design department was spared the trouble. There was an independent designer who had access to our president and he showed him a collection of [Arctic] designs. There were two things wrong: (1) it was too early because the collection hadn’t hit town yet; (2) more important, the designs themselves were horrible. Anyway, [the collection] was shown in a room with everything spread out on a large table. I was called down to this room which was crowded with about nine people from the company who had seen the designs. I looked at this display and instantly hated them. I was asked what I thought but before I could open my mouth, people were jumping up and down clapping the designer on the back and so on. They had already decided to do it because the president had loved it. Of course, the whole affair was a total failure. The point is that in making decisions, people look up and look around. They rely on others, not because of inexperience, but because of fear of failure. They look up and look to others before they take any plunges. (Location 1713)

112. But all these things have no relationship to the way they actually manage or make decisions. The basic principles of decision making in this organization and probably any organization are: (1) avoid making any decision if at all possible; (2) if a decision has to be made, involve as many people as you can so that, if things go south, you’re able to point in as many directions as possible. (Location 1731)

113. A middle-aged, upper-middle level manager at Alchemy Inc. says: You know, there is this huge computerized inventory of skills which people update each year; it’s called a skills inventory. … But all the computerized lists in the world don’t amount to much in the corporation. What matters is a bunch of guys sitting informally in a room and deciding who should get jobs and who shouldn’t. The real job decisions are made on that basis. And circumstances determine your fate. (Location 1634)

114. Making a decision, or standing by a decision once made, exposes carefully nurtured images of competence and know-how to the judgments of others, particularly of one’s superiors. As a result, many managers become extremely adept at sidestepping decisions altogether and shrugging off responsibility, all the while projecting an air of command, authority, and decisiveness, leaving those who actually do decide to carry the ball alone in the open field. (Location 1774)

115. This explains why the chemical company managers kept putting off a decision about major reinvestment. After the battery collapsed in 1979, however, the decision facing them was simple and posed little risk. The corporation had to meet its legal obligations; also, it had either to repair the battery the way the EPA demanded or shut down the plant and lose several hundred million dollars. Since there were no real choices, everyone could agree on a course of action because everyone could appeal to inevitability. This is the nub of managerial decision making. As one manager says: Decisions are made only when they are inevitable. To make a decision ahead of the time it has to be made risks political catastrophe. People can always interpret the decision as an unwise one even if it seems to be correct on other grounds. (Location 1886)

116. Despite their fresh appearances, certain themes recur constantly in the programs offered by consultants. Perhaps the most common are how to sharpen decision making, how to restructure organizations for greater efficiency, how to improve productivity, how to recognize trouble spots in an organization, how to communicate effectively, how to humanize the workplace, and how to raise morale. (Location 3164)

J. This is Your Life

117. At the managerial and professional levels, the road between work and life is usually open because it is difficult to refuse to use one’s influence, patronage, or power on behalf of another regular member of one’s social coterie. It therefore becomes important to choose one’s social colleagues with some care and, of course, know how to drop them should they fall out of organizational favor. (Location 884)

118. For some managers, the drive for success is a quest for the generous financial rewards that high corporate position brings. For others, success means the freedom to define one’s work role with some latitude, to “get out from under the thumb of others.” For still others, it means the chance to gain power and to exert one’s will, to “call the shots,” to “do it my way,” or to know the curiously exhilarating pleasure of controlling other people’s fates. For still others, the quest for success expresses a deep hunger for the recognition and accolades of one’s peers. (Location 955)

119. More generally, those who accept immobility are unwilling to sacrifice family life or free-time activities to put in the extraordinarily long hours at the office required in the upper circles of their corporations. Or they have made a realistic assessment of the age structure, career paths, and power relationships above them and conclude that there is no longer real opportunity for them. They may see that there is an irreparable mismatch between their own personal styles and the kinds of social skills being cultivated in well-entrenched higher circles. In many cases, they decide that they do not wish to put up with the great stress of higher management work that they have witnessed. (Location 962)

120. Higher-level managers in all the corporations I studied commonly spend twelve to fourteen hours a day at the office. (Location 1156)

121. In a world where appearances—in the broadest sense—mean everything, the wise and ambitious manager learns to cultivate assiduously the proper, prescribed modes of appearing. He dispassionately takes stock of himself, treating himself as an object, as a commodity. He analyzes his strengths and weaknesses and decides what he needs to change in order to survive and flourish in his organization. And then he systematically undertakes a program to reconstruct his image, his publicly avowed attitudes or ideas, or whatever else in his self-presentation that might need adjustment. (Location 1339)

122. This [lack of control] doesn’t mean you don’t work hard; at least in my case, that’s my answer. I have to believe I can influence events. That way, I feel good about myself even if my boss doesn’t. (Location 1606)

123. Sometimes I’ll wake up in the middle of the night thinking about the plant. And if I can’t get back to sleep, I’ll slip out of bed and walk over to the plant and just walk around the machinery and talk to the guys. I love the smell of the oil and the grease and the sound of the machines. For me, that’s what life is all about. (Location 4535)

K. Immoral Mazes

124. In analyzing incidence data from a representative plant, and by extrapolating to the rest of the firm’s mills, White discovered that 12 percent of all greige mill workers had already suffered hearing loss severe enough to be immediately compensable under state law for as much as $3.5 to $5.7 million. This, however, was only the thunder before a summer storm. Another 63 percent of greige mill workers had already suffered substantial, though not yet compensable, damage that could only worsen the longer they stayed in the industry. (Location 2265)

125. Moreover, by insisting on his own personal moral purity, his feeling that if he did not expose things he himself would be drawn into a web of corruption, he was, they feel, being disingenuous; no one reaches his level of a hierarchy without being tainted. (Location 2450)

126. A moral judgment based on a professional ethic makes little sense in a world where the etiquette of authority relationships and the necessity for protecting and covering for one’s boss, one’s network, and oneself supercede all other considerations and where nonaccountability for action is the norm. (Location 2474)

127. On the polar crane issue, the top official of the NRC later characterized Wilson’s and his engineers’ concerns as stemming from a philosophy that emphasized procedural matters rather than a focus on final goals. This characterization was echoed by GPUN management who stressed that what was at issue in the polar crane dispute was not procedures but results; at a certain point, they said, decisions had to be made to resolve technical disputes and work had to proceed toward what everyone acknowledged to be a worthwhile goal, that is, the cleanup of TMI-2. (Location 2566)

128. Once, when he wrote a memo to the GPUN deputy director about radioactively contaminated sewage being trucked out of the plant and disposed of illegally, his boss replied that he did not need such a memo from Wilson. It was, his boss said, not constructive and wasted his own and Wilson’s time. (Location 2575)

129. my view, this is the nub of the moral ethos of bureaucracy. Managers see this issue as a “trade-off” between principle and expediency. They usually pose the trade-off as a question: Where do you draw the line? (Location 2652)

130. It is said, in fact, that one Weft manager who was on the take for years from customers was fired within a half hour of the discovery of his “inexplicably stupid” acceptance and deposit of a check from his benefactors instead of his normal cash rake-offs. With such authoritative encouragement, managers internalize these norms into their own personal codes of honor; they speak privately of the importance of not being known as men or women “who can be had.” (Location 2658)

131. It gets hard. Now, suppose that the ozone depletion theory were correct and you knew that these specific fifty people were going to get skin cancer because you produced chlorofluorocarbons [CFCs]. Well, there would be no question. You would just stop production. But suppose that you didn’t know the fifty people and it wasn’t at all clear that CFCs were at fault, or entirely at fault. What do you do then? (Location 2831)

132. Suppose that you had a candy bar factory and you were touring the plant and you saw with your own eyes a worker slip a razor blade into a bar. And before you could stop the machine, there were a thousand bars more made and the one with the razor blade was mixed up. Well, there’s no question that you would get rid of the thousand candy bars. But what if it were a million bars? Well, I don’t know what I’d do. (Location 2843)

133. Moreover, he tries to treat his subordinates forthrightly, firmly believing that one’s word is an important measure of a person. In a world, however, where actions are separated from consequences, where knowledge is fragmented and secreted, where private agreements are the only real way to fashion trust in the midst of ongoing competition and conflict, where relationships with trusted colleagues constitute one’s only real means both of defense and opportunity, and where, one knows, even coincidental association with a disaster can haunt one’s career years later, keeping silent and covering for oneself and for one’s fellows become not only possible but prudent, indeed virtuous, courses of action. (Location 2969)

134. Moreover, it is a byword in the chemical industry at least that it is precisely in those technological areas where accidents have seldom occurred that the largest potential catastrophes loom; the very lack of practice in responding quickly to untoward incidents can precipitate uncontrollable events. (Location 3376)

135. Publicly, of course, Weft Corporation, as do many other firms, claims that the money was spent entirely to eliminate dust, evidence of its corporate good citizenship. Privately, executives admit that without the productive return, they would not have—indeed, given the constraints under which they operate—could not have spent the money. (Location 3585)

136. The ethic that emerges in bureaucratic contexts contrasts sharply in many respects with the original Protestant ethic. The Protestant ethic was a social construction of reality of a self-confident and independent propertied social class. It was an ideology that extolled the virtues of accumulating and reinvesting wealth in a society organized around property and that accepted the stewardship responsibilities entailed by property. It was an ideology where a person’s word was his bond and where the integrity of the handshake was crucial to the maintenance of good business relationships. (Location 4295)

137. At the core of the middle class’s righteous, some would say smug, faith in itself, of its inexhaustible drive, of its unremitting pragmatism, was the conviction that hard work necessarily had its just rewards here and now as a token of divine favor in the hereafter. This conviction was also the bedrock of a profound guilt mechanism that impelled one to fulfill personal and social obligations; failure to do so, like a failure to work hard, was thought to be a sin against both God and self. (Location 4304)

138. Bureaucracy breaks apart the ownership of property from its control, social independence from occupation, substance from appearances, action from responsibility, obligation from guilt, language from meaning, and notions of truth from reality. Most important, and at the bottom of all of these fractures, it breaks apart the older connection between the meaning of work and salvation. In the bureaucratic world, one’s success, one’s sign of election, no longer depends on an inscrutable God, but on the capriciousness of one’s superiors and the market; and one achieves economic salvation to the extent that one pleases and submits to new gods, that is, one’s bosses and the exigencies of an impersonal market. (Location 4307)

L. Truth and Public Relations

139. Truth? What is truth? I don’t know anyone in this business who talks about “truth.” (Location 4137)

140. After all, as one executive says, “We lie all the time, but if everyone knows that we’re lying, is a lie really a lie?” (Location 2693)

141. If a retailer returns with goods and says, “Look, I can’t sell $50,000 of this stuff. You have to take it back or it’s going to break me,” may one say, “What’s that? I didn’t hear you,” hoping for the sake of future business that the retailer will say, “Oh, look, you were late delivering and I can’t use it now.” (Location 2700)

142. The consultant who perceives such discrepancies has to devise his own strategies for handling them. Some of these include: rejecting the assignment altogether; accepting the problem as defined and confining oneself to it for the sake of future contracts even though one knows that any action will be inefficacious; or accepting the assignment but trying to persuade the client to address the underlying social and political issues, that is, redefining the problem. (Location 3234)

143. For instance, a highly placed staff member whose work requires him to interact daily with the top figures of his company, says: I get faked out all the time, and I’m part of the system. I come from a very different culture. Where I come from, if you give someone your word, no one ever questions it. It’s the old hard-work-will-lead-to-success ideology. Small community, Protestant, agrarian, small business, merchant-type values. I’m disadvantaged in a system like this. He goes on to characterize the system more fully and what it takes to succeed within it: It’s the ability to play this system that determines whether you will rise. … And part of the adeptness [required] is determined by how much it bothers people. One thing you have to be able to do is to play the game, but you can’t be disturbed by the game. What’s the game? It’s bringing troops home from Vietnam and declaring peace with honor. It’s saying one thing and meaning another. It’s characterizing the reality of a situation with any description that is necessary to make that situation more palatable to some group that matters. It means that you have to come up with a culturally accepted verbalization to explain why you are not doing what you are doing…. [Or] you say that we had to do what we did because it was inevitable; or because the guys at the [regulatory] agencies were dumb; [you] say we won when we really lost; [you] say we saved money when we squandered it; [you] say something’s safe when it’s potentially or actually dangerous…. Everyone knows that it’s bullshit, but it’s accepted. This is the game. (Location 3246)

144. Such adeptness at inconsistency, without moral uneasiness, is essential for executive success. Done over a period of time, in fact, it seems to become a taken for granted habit of mind. As one executive at Covenant Corporation says: Now some people don’t understand this…. But as you move up the ladder, you don’t have people who don’t understand. And the people up high don’t necessarily do it consciously. They are able to speak out of both sides of their mouth without missing a step. It means being able to say, as a very high-ranking official of Weft Corporation said to me without batting an eye, that the industry has never caused the slightest problem in any worker’s breathing capacity. It means, in Covenant Corporation, propagating an elaborate hazard/benefit calculus for approval of dangerous chemicals while internally conceptualizing “hazards” as business risks. It means publicly extolling the carefulness of testing procedures on toxic chemicals while privately ridiculing animal tests as inapplicable to humans. (Location 3610)

145. Finally, those truly adept at inconsistency can also interpret with some accuracy the inconsistent machinations of their colleagues and adversaries. This is not a mean skill. At the very beginning of my fieldwork, the top lawyer of a large corporation was discussing an issue that I had raised when he said: Now, I’m going to be completely honest with you about this. He paused for a moment and then said: By the way, in the corporate world, whenever anybody says to you: “I’m going to be completely honest with you about this,” you should immediately know that a curveball is on the way. But, of course, that doesn’t apply to what I’m about to tell you. (Location 3631)

146. Since the success of large commercial bureaucracies depends to a great extent on the goodwill of the consuming public, ambitious managers recognize that great organizational premiums are placed on the ability to explain expedient action convincingly. Public opinion, of course, constitutes one of the only effective checks on the bureaucratic impulse to translate all moral issues into practical concerns. (Location 3643)

147. 1925, Walter Lippmann critically analyzed a public’s ability to appreciate the intricacies of any issue or indeed to keep its attention on anything but crises. He adds: [S]ince [a public] acts by aligning itself, it personalizes whatever it considers, and is interested only when events have been melodramatized as a conflict. The public will arrive in the middle of the third act and will leave before the last curtain, having stayed just long enough perhaps to decide who is the hero and who is the villain of the piece. (Location 3765)

148. read in The Wall Street Journal or The New York Times accounts of specific events or larger trends attributed to “informed sources,” the code name for public relations men and women; (Location 3828)

149. There are, of course, good clients and bad clients. A good client keeps his public relations specialists adequately informed, provides “feedback” and “constructive criticism,” and recognizes that public relations depends on time-billing rather than on product billing and provides enough funds to do the job properly. A good client “takes stock of himself,” that is, dispassionately objectifies his company’s products, his company’s organizational structure and personnel, and, say, in the case of a top corporate official, himself, to make them all more readily manipulable and therefore marketable commodities. Above all, a good client is flexible and therefore able rapidly to shift ground and actual policies as well to meet new needs or new pressures. A bad client, by contrast, either does not understand or chooses to ignore the peculiarly indirect approach of public relations and wants immediately observable results in terms of press clippings or TV time; uses a public relations program as a vehicle for self-aggrandizement within his own corporation, placing the public relations specialist in danger of “getting caught in a pissing contest between executives”; demands the release of press statements that are only marginally “newsworthy” and then blames the public relations specialist when nothing at all gets printed or aired; conceals crucial aspects of his story from his public relations advisers and refuses them adequate access to his staff and facilities to get the full story; comes to the public relations agency with trumped-up data and fake photographs—as happened, for instance, at Images Inc. when a client falsely claimed the efficacy of a product to remove tar from despoiled beaches—and ends up declaring bankruptcy, leaving the public relations agency liable for a multimillion-dollar lawsuit; wants the public relations agency, as in the case of some single-party foreign governments, to put a good public face on practices like the “persuasion” and “elimination” of opponents; expects the public relations agency to be the “bag man” to pay off government officials or newspaper editors; and, perhaps especially, insists on an indefensible or totally unbelievable version of reality or expects the public relations specialist to tell outright lies. (Location 3843)

150. As it happens, accounts in the public relations world circulate continually and only sometimes because of actual or alleged dissatisfaction with public relations service. The pattern of continual mergers, upheavals, and power struggles in corporations, that I have argued earlier is at the core of American corporate life, directly affects public relations agencies. When a new CEO or divisional president assumes power in a corporation, he will quite often change public relations and advertising agencies as part of a larger strategy of shedding the past or to assert his own “new vision” of the future. When this occurs, almost invariably, he will move his account to an agency whose leaders he knows and with whom he feels comfortable. This continual circulation of accounts means that agency personnel are constantly searching for new accounts and constantly devising ways to hold present clients, despite the knowledge of the inevitability of their eventual departure. (Location 3868)

151. In the world of public relations, there is no such thing as a notion of truth; there are only stories, perspectives, or opinions. One works, of course, with “facts,” that is, selected empirically verifiable statements about the world. But as long as a story is factual, it does not matter if it is “true.” One can feel free to arrange these facts in a variety of ways and to put any interpretation on them that suits a client’s objectives. Interpretations and judgments are always completely relative. The only canons binding this process of interpretation are those of credibility or, more exactly, of plausibility. If an interpretation of facts, a story, is taken as plausible by a targeted audience, it is just as good as “true” in any philosophical sense, indeed better since it furthers the accomplishment of an immediate goal. (Location 3886)

152. From the standpoint of public relations, the journalistic ideology closely resembles the social outlook of most college seniors—a vague but pious middle-class liberalism, a mildly critical stance toward their fathers in particular and authorities in general; a maudlin championship of the poor and the underclass; and especially the doctrine of tolerance, open-mindedness, and balance. In fact, public relations people feel, the news media are also constructing reality. They are always looking for a “fresh” and exciting angle; they have an unerring instinct for the sentimental that expresses itself in a preference for “human interest” rather than substance; and they arrange facts in a way that purports to convey “truth,” but is in fact simply another story. In reality, news is entertainment. And, despite the public’s acceptance of journalistic ideologies, most of the public watch or read news not to be informed or to learn the “truth,” but precisely to be entertained. There is no intrinsic reason, therefore, why the constructions of reality by public relations specialists should be thought of as any different from those of any group in the business of telling stories to the public. (Location 3903)

153. The top official, in a written document prepared for a meeting to discuss the issue, argued that: [T]he final report was better than it would have been if it had not gone through [the] process of reexamination. It was stronger, it was more important, it was more constructive…. At the same time, there is no doubt that pressures were brought on us to come to the kind of conclusions we finally reached. At the meeting, he added, somewhat more pithily: “We tried to be honest but, believe me, it wasn’t easy.” (Location 4124)

154. The same top executive defines truth: I sometimes sit back and think that if we could make up a list of all the viewpoints of all our clients and somehow fit them together, then that would be truth. That would be what we are as a firm. (Location 4130)

155. Agency practitioners not only defend multiple interests but face repeatedly the experience of even well-served clients switching agencies. Especially unsettling are the departures of clients who decide that, since they are as they have been portrayed, they have no future need to construct reality. The continual circulation of client accounts in agencies diminishes the possibilities of comforting, long-held allegiances to organizations, products, or causes. (Location 4192)

156. Alternatively, and by contrast, practitioners in both settings sometimes justify their efforts by appealing to a professional ethos that celebrates the exercise of technical skill separated from any emotional commitment to one’s clients. A dignified version of this legitimation is the often repeated analogy between public relations practitioners and lawyers; both occupations, it is argued, fulfill important advocacy roles in a free society. Only the practice of the professional virtue of public relations, however hazardous to individual practitioners, can assure the continued diversity of opinion that marks our democracy. (Location 4195)

157. That’s the reality of our society. There’s no question that their story is being told because they have the money and power. I’ve got to recognize that I’m part of this society and just come to live with that. Our society is the way it is. It’s run on money and power, it’s that simple. Truth has nothing to do with it. So we just accept the world as it is and live with it. (Location 4206)

158. The agency literally creates these realities by matchmaking artistic talent and accomplishment with money. In the process, up-and-coming junior corporate executives get the kind of exposure to refined, sophisticated artistic and intellectual circles that will help prepare and polish them for higher posts. Artists, in turn, as long as their work is not too avant-garde, receive the benefits of a latter-day Medici-like patronage. Public relations people claim a double accomplishment—they help civilize businessmen, not least by inspiring in them a “passion for greatness” rather than self-interest as the important motive for patronage of the arts; and they provide the public with access to high culture. The same firm has also arranged corporate sponsorship of dance and musical performances, helped develop important educational programs such as one on infant nutrition, conducted some useful surveys such as one on the problems facing ethnic minorities, and done a lot of pro bono work for philanthropic, community, and public service organizations in the bargain. One tries, then, to move some clients in directions that seem socially desirable while at the same time playing with the magic lantern to serve their interests. Sometimes, too, the ability to accomplish any good at all in this world seems to depend on the willingness to serve even clients with no apparent redeeming features in order to seize capricious opportunities to channel other clients’ resources into work deemed socially worthwhile. (Location 4214)

159. In short, bureaucracy creates for managers a Calvinist world without a Calvinist God, a world marked with the same profound anxiety that characterized the old Protestant ethic but one stripped of that ideology’s comforting illusions. Bureaucracy poses for managers an intricate set of moral mazes that are paradigmatic of the quandaries of public life in our social order. Within this framework, the puzzle for many individual managers becomes: How does one act in such a world and maintain a sense of personal integrity? (Location 4351)

M. Under Siege

160. Part of the folklore of the modern corporation, in fact, consists of a catalog of stories about how big corporations are being victimized by the courts in what amounts to a radical transformation of tort law. Managers repeat variations of cases—for instance, of a man who purchased a thrice-owned punch machine, altered it, and after losing a finger, sued the original manufacturer for negligent construction; of a farmer who, despite clearly marked warning labels, coated his animal feed troughs with a toxic wood preservative, sold milk from contaminated cattle, and then when he himself was sued by irate parents whose children had drunk the milk, sued the chemical company who produced the preservative; or of a student research assistant inexplicably asked to carry dangerous acid in glass containers on a faulty elevator that lurched and caused the young woman to break a container, showering her with acid and permanently disfiguring her. Although she is said to have sued her university, her supervising professor, the elevator manufacturer, and the chemical company that provided the acid, managers had few doubts that the jury would “pin the tail on business” because of the “abiding conviction that corporations have vaults filled with gold bars called profits.” At some level, even managers who repeat such stories and thus reinforce their own shared sense of beleaguerment, recognize the profound and profoundly felt dependence of most people in our society on those large organizations that claim to control the scientific genie. (Location 3413)

161. was at this party the other night and I was sitting next to this older lady and she said: “My God, did you people see the paper tonight? There’s leakage from some chemical plant and it’s infecting the drinking water around here.” So I asked if I could see the paper and the article said that there was some seepage out of a pond that a chemical plant used for disposal but that the EPA was monitoring the situation as well as all 25 wells in the area. In the meantime, the company was remedying the situation. I pointed this out and she looked at me, her eyes narrowed, and she asked, “Who do you work for?” I said I worked for a large chemical firm and she burst out laughing and asked if I expected her to believe me. She laughed right in my face! I asked her what she was worried about. You have to drink the water for 25 years before anything would happen—I mean, she was already a grandmother—but that didn’t seem to help her much. (Location 3459)

162. My interviews are filled with stories of managers who claim to have been verbally assaulted not only by strangers at cocktail parties, but by their children’s teachers when they visit schools, and even by their children themselves at the breakfast table for being supposedly callous and insensitive to the social consequences of business activity. Perhaps more than anything else, managers are puzzled by such attacks, though they pounce quickly on any inconsistencies that they perceive in their opponents. For example, one manager whose firm produced a pesticide that became caught up in a widely publicized episode of mishandling and illegal disposal was attacked by his brother-in-law, a lifelong military man, for his very association with the company. The manager found it grimly ironic that “things have reached a point where a trained killer is berating me for producing something useful.” More generally, managers view the irrational fear of contamination evinced by those who espouse an ideology of bodily purity with some derision. One can, after all, referring to the same pesticide, “eat handfuls of the stuff with no lasting adverse effects.” (Location 3467)

163. Well, from 1957 through 1962, I was intimately involved with the manufacture of DDT. During that time, we doubled production and sold almost all of it to Africa and India. And I knew and went home knowing that I was saving more lives than any major hospital was capable of doing. I knew that I was saving thousands of lives by doing this. Then Rachel Carson’s Silent Spring came out and not only did I become a murderer of falcons and robins, but also one of the mass murderers of the world. I was now doing evil things to the world. Then I went to a plant which was manufacturing [chlorofluorocarbons] and we increased production by 20–25 percent; a lot of it went into hair sprays. We also used vinyl chloride and found out that it was causing liver cancer. Then I found out that I was destroying the whole ozone layer of the earth and doing it for personal gain. Then I went into soda ash. Without it, there wouldn’t be a window pane in the whole United States. But at the time, because of the Clean Air Act and the Clean Water Act, suddenly I became a polluter. Children learned in school that chemicals killed. And, of course, there was no question in the academic community that I was perceived as an evil person doing evil things. And that became true even in the corporation. Plants became a liability, rather than the source of wealth. The perception was: Wouldn’t it be nice if we could just sell chemicals without producing them? So the profession of producing things became a low profession and the good people were those who were producing services. Manufacturing people became evil. I think this is one reason that marketing people became ascendant in the competition for advancement. Then I went into making sulfuric acid and that involved the whole issue of water pollution. He goes on to describe his reaction to all of this: I guess I’m more bemused than anything else. It’s the same type of feeling that I would have if I were an MD who had been doing radical mastectomies and then someone says—hey, you didn’t have to do that. It’s a feeling of disappointment without a feeling of shame. I know that I have done some useful things. I know that the only source of money is taking something out of the ground and making something. That’s where money comes from—making something out of nothing. Without that type of activity, civilization wouldn’t exist. So, on something like DDT, I’ll leave the judgment to Europe after World War II and all the people saved by the widespread use of it. (Location 3482)

164. Within this context of perceived harassment and shifting scientific and ideological winds, while always attending to the pressing and sometimes contradictory exigencies of business life, managers must address a multiplicity of audiences, some of whom are considered rivals, and some outright adversaries. These audiences are the internal corporate hierarchy with its intricate and shifting power cliques and competing managerial circles, key regulators, local and federal legislators, special publics that vary according to the issues, and the public at large, whose goodwill and favorable opinions are considered essential for a company’s free operation. Managerial adeptness at inconsistency becomes evident in the widely discrepant perspectives, reasons for action, and presentations of fact that explain, excuse, or justify managerial behavior to these diverse audiences. (Location 3506)

165. The Supreme Court, however, decided in 1981 that the 1978 OSHA ruling was fully within the Agency’s mandate, namely to protect workers’ health and safety as the primary benefit exceeding all cost considerations. (Location 3533)

166. The segmented work patterns of bureaucracy underlie these larger structures. Managers’ cognitive maps to the thickets of their world contain sharp, sometimes absurd, caricatures of the style and ethos of different occupational groups. These suggest some of the ways in which managers appraise the myriad of character types whom they see peopling their world. Production types, for instance, are said to be hard-drinking, raucous, good-time charlies; engineers, always distinguishable by the plastic pen containers in their shirt pockets, are hostages to an outdated belief in a pristine mathematical rationality; accountants are bean counters who know how to play the shell game; lawyers are legal eagles or legal beagles in wool pinstripes who, if they had their way, would tie managers’ hands completely; corporate staff are the king’s spies, always ready to do his bidding and his dirty work; marketing guys are cheerful, smooth-talking, upbeat fashion plates who must nonetheless keep salesmen under their thumbs; salesmen are aggressive loudmouths who feel that they can sell freezers to penguins in Antarctica and who would sell their grandmothers just to make a deal. Salesmen hate the restraints that marketers put on their work and on their ego gratification. Financial wizards, on the ascendancy everywhere, are tight-mouthed, close to the vest poker players who think that a social order can be built on paper deals. And outside consultants are men and women who borrow one’s watch and then charge for telling the time. (Location 4327)

167. But so what, they argued, if the preservative did in fact pose some risk of cancer? Better, they said, the risk of a slight long-run increase in the rate of stomach and intestinal cancer than the certainty of a precipitous spurt in the incidence of botulism, particularly in the lower-income black and Hispanic groups that typically consume large amounts of processed meat and, both because of poverty and cultural practices, often leave food uncovered and unrefrigerated for considerable periods. Is corporate social responsibility, they asked, maintaining a private sense and public image of moral purity while someone else does necessary but tainted work? Or is real social responsibility the willingness to get one’s hands dirty, to make whatever compromises have to be made to produce a product with some utility, to achieve therefore some social good, even though one knows that one’s accomplishments and motives will inevitably be misinterpreted by others for their own ends, usually by those with the least reason to complain? Besides, they pointed out, consumers continue to purchase artificially preserved meats in large quantities. Is not the proper role of business “to give the public what it wants,” adopting the market as its polar star, as the only reliable guide in a pluralistic society to “the greatest good for the greatest number,” as the final arbiter not of values, which are always arguable, but, more importantly, of tastes, about which there can be no reasonable dispute? (Location 4500)

168. In fact, exercises in substantive rationality—the critical, reflective use of reason—are not only subject to infinite interpretations and counterinterpretations but also invite fantastic constructions of reality, including attributions of conspiracy. Thus a major corporation provides a gift of $10 million to establish new foundations that will materially aid South African blacks and is promptly accused by a black American leader of bolstering apartheid. Weft managers create an elaborate recreational complex for Weft employees in the corporation’s southern community and are charged with perpetuating traditional textile company paternalism. Some executives at Images Inc. donate their time to bring together several institutional sectors of a local town in which they live for community betterment and are charged with trying to grab headlines and line up future business. Managers often feel that, however genuine it may be, altruism is a motive that is always denied them by others. To complicate matters still further, the necessary self-promotional work of presenting private goals as public goods, or the self-defensive work within the corporation of presenting public goods as hardheaded business decisions, or managers’ knowledge that bureaucracy insulates them from the real consequences of their actual choices, often make their protestations of socially responsible actions suspect even to themselves. (Location 4512)


OECD agenda [Link]

30 мая, 2019 - 12:51
Published on May 30, 2019 9:51 AM UTC


What's your oppinions about this?


Archive of all LW essay contests

30 мая, 2019 - 09:40

How to find a lost phone inside your own home, using Google Location History Takeout

30 мая, 2019 - 07:56
Published on May 30, 2019 4:56 AM UTC

Or, wow, it's crazy how much Google knows about me.

(I'm writing this in part because I tried to Google "how to find a lost phone inside home" and among all the articles in the search results that I clicked on, didn't see anyone write up this technique that I later figured out myself.)

I lost a brand new Android phone recently inside my own home. I knew it was inside my home because I remembered using it after I got home. Normally in that situation I would call it or use Google's Find Device feature to make the phone ring, but the phone's battery had died before I realized that it was lost. I spent a couple of hours looking for it in all the obvious places, like behind sofa cushions, in the garbage can, everywhere in the house I had been (or might have been but forgot) since I last used the phone, but failed to find it. Find Device gives the last known location of a phone, but the error circle encompassed the whole house so that wasn't very helpful.

Eventually I decided to download all of the data Google has on the history of the phone's location using a feature called Takeout, just in case it offered any further clues, and that's when I realized that in Google's databases there's an entry for every time one of my phones moves even a little bit, with a timestamp, location estimate, velocity estimate, and a guess of whether I was walking, running, biking, in a car, etc. (presumably inferred using the phone's accelerometer). I converted the data from JSON to CSV and then looked at the last time the phone moved. Then I looked at my Google Activity History to find out what I was doing at that time. The Activity History told me I did a particular Google search around that time, and I remembered I was sitting in a particular armchair while doing that search. (Google Activity History also has a whole bunch of other information, like every time I launch an app on my phone, or visit a website using Chrome.)

Of course I had searched that armchair before but this knowledge incentivized me to do a much more thorough search, and it turned out that the phone had fallen down a very narrow crack in the armchair to a pocket underneath. I ended up retrieving the phone by cutting open the fabric covering the bottom of the armchair.


South Bay Meetup

30 мая, 2019 - 05:32
Published on May 30, 2019 2:32 AM UTC


Seeking suggestions for EA cash-prize contest

30 мая, 2019 - 00:48
Published on May 29, 2019 8:44 PM UTC

I have grown interested in the cash-prize EA essay contest format that I've seen on LessWrong. I'm interested in sponsoring a 3-essay series of contests, and would appreciate exploratory thoughts and critique. My goal is to make this essay series produce genuinely useful new information for the community and lead to positive changes in the way we organize these contests. Any strategic advice would be helpful.

The first essay would be on the prompt "What is the most effective way to run a cash-prize EA essay contest? Make sure to specify how winners should be selected and prize money distributed."

The second would implement suggestions from the first contest, and be on the prompt "what is the most compelling argument for not running cash-prize EA essay contests?"

The third would be on the prompt "How could we design a useful scientific experiment to determine the ideal size of essay cash prizes?"

My plan is to offer $50-$100 as cash prizes for each essay. I don't earn a lot by American standards, which is why these prizes are relatively small. I'm considering crowdfunding for higher prizes. I'm not sure if I should allocate it all to the winner or divide it among top posts.

For the second essay, I can determine the winner through the method suggested by the first essay. But for judging the first essay, how should I go about it?


What is a good moment to start writing?

30 мая, 2019 - 00:47
Published on May 29, 2019 9:47 PM UTC

I notice that I never have a complete answer to anything.

I may have digged into something way more than my peers, but there are always all kinds of hunches I haven't put into words yet, all kinds of data that I haven't internalized yet, all kinds of unresolved confusions.

I know much more than a year ago. If I explain my ideas now, I'm going to be embarrassed by it next year.

My opposite worry is that I will learn too much. Forget how I learned all this stuff, at which point the inferential distance is too large to overcome.

This worry is not about impressions. It's about not wasting people's time with half-baked ideas. Naively one might just max out the tree of details before reporting them, but the tree is never maxed out. There is always ambiguity somewhere. How much is acceptable to pass on?


What is required to run a psychology study?

29 мая, 2019 - 09:38
Published on May 29, 2019 6:38 AM UTC

I periodically find myself wishing someone had run an experiment on a particular topic. But they haven't.

Often, it seems like there's a relatively easy experiment you can run, which would give me some evidence about it. Maybe it wouldn't be perfect evidence, but it would be better than me making stuff up from my armchair.

The two clusters of things-I-can-imagine-being-quite-hard are:

  • Legal requirements
  • Actually doing the science right (avoiding sampling biases, actually testing the right variable, etc)

Legal Stuff

Scott's IRB Nightmare suggests that the legal requirements can be quite awful. I'm unclear on what those requirements are if I don't want to publish anywhere, don't plan to interface with any hospital bureaucracies, I just want to ask a bunch of people to try something and see how it goes (and maybe make a LessWrong blogpost so people on LessWrong can know too)

Am I allowed to just go out and ask a whole bunch of people stuff? If I want to know things about 8 year olds or 16 year olds, if their parents sign a form and I'm not doing anything especially weird or traumatic, would that be fine or would terrible things happen to me? (Could I replicate the Marshmallow Test?)

Doing Science Good Enough

I have a sense that there's a lot of ways to deceive yourself if you have a pet psychology theory. But... I dunno it also seems like LessWrong collectively should be pretty good at this. The thing I might want to do (or get others to do sometimes) is post a plan for a study here, and get critiqued until it seems like an actually good plan, do the plan, write up the results.

Particular Things I was interested in:

(not meant to be exhaustive, important, or particularly achievable plans, just the things that generated this question)

  • How do most people relate to their job? (There's a survey study that asks if people consider it a job, a career, or a calling, but the study had a very narrow range of participants – people who worked at one particular place, and I'm curious how a wider variety of people would respond)
  • How do most people relate to ambition? (where by ambition I mean "form plans to create or change things that will impact large numbers of people.")
  • How many 8 year olds can learn to program? Can they implement FizzBuzz? How many 16 year olds? How many 30 year olds? I've heard that people either have-or-don't-have a "programmer trait" that's hard to learn, and I'm not sure how much of that is true at all, and if so if it's more about nature or nurture.


MIRI Summer Fellows Program

29 мая, 2019 - 03:28
Published on May 29, 2019 12:28 AM UTC

CFAR and MIRI are running our fifth annual MIRI Summer Fellows Program (MSFP) in the San Francisco Bay Area from August 9 to August 24, 2019.  

MSFP is an extended retreat for mathematicians and programmers with a serious interest in making technical progress on the problem of AI alignment.  It includes an overview of CFAR's applied rationality content, a breadth-first grounding in the MIRI perspective on AI safety, and multiple days of actual hands-on research with participants and MIRI staff attempting to make inroads on open questions.

Program Description

The intent of the program is to boost participants, as far as possible, in four overlapping areas:

Doing rationality inside a human brain: understanding, with as much fidelity as possible, what phenomena and processes drive and influence human thinking and reasoning, so that we can account for our own biases and blindspots, better recruit and use the various functions of our brains, and, in general, be less likely to trick ourselves, gloss over our confusions, or fail to act in alignment with our endorsed values.

Epistemic rationality, especially the subset of skills around deconfusion. Building the skill of noticing where the dots don't actually connect; answering the question "why do we think we know what we think we know?", particularly when it comes to predictions and assertions around the future development of artificial intelligence.

Grounding in the current research landscape surrounding AI: being aware of the primary disagreements among leaders in the field, and the arguments for various perspectives and claims. Understanding the current open questions, and why different ones seem more pressing or real under different assumptions. Being able to follow the reasoning behind various alignment schemes/theories/proposed interventions, and being able to evaluate those interventions with careful reasoning and mature (or at least more-mature-than-before) intuitions.

Generative research skill: the ability to make real and relevant progress on questions related to the field of AI alignment without losing track of one's own metacognition.  The parallel processes of using one's mental tools, critiquing and improving one's mental tools, and making one's own progress or deconfusion available to others through talks, papers, and models.  Anything and everything involved in being the sort of thinker who can locate a good question, sniff out promising threads, and collaborate effectively with others and with the broader research ecosystem.

Food and lodging are provided free of charge at CFAR's workshop venue in Bodega Bay, California.  Participants must be able to remain onsite, largely undistracted for the duration of the program (e.g. no major appointments in other cities, no large looming academic or professional deadlines just after the program).

[5/28/19 Update: Applications closed on March 31, finalists were interviewed between April 1 and April 17, and admissions decisions (yes, no, waitlist) were sent in April.]

If you have any questions or comments, please email Colm at the contact address above, or, if you suspect others would also benefit from reading the answer, post them here.


Drowning children are rare

28 мая, 2019 - 20:19

[Link] Evidence for Connection Theory

28 мая, 2019 - 20:06
Published on May 28, 2019 5:06 PM UTC

This is a linkpost for Connection Theory: The Current Evidence.

Connection Theory (CT) is the original philosophy underpinning Leverage Research, a research think tank focused that has worked with the effective altruism movement, and the rationality community, in the past on community-building, and existential risk reduction. CT was developed by Leverage's executive director, Geoff Anders. Since there are few if any other publicly available online resources for understanding or evaluating CT, I thought I would share this document.


A shift in arguments for AI risk

28 мая, 2019 - 16:47
Published on May 28, 2019 1:47 PM UTC

The linked post is work done by Tom Sittler while at FHI. I think this sort of expository and analytic work is very valuable, so I'm cross-posting it here (with his permission). Below is an extended summary; for the full document, see his linked blog post.

Many people now work on ensuring that advanced AI has beneficial consequences. But members of this community have made several quite different arguments for prioritising AI.

Early arguments, and in particular Superintelligence, identified the “alignment problem” as the key source of AI risk. In addition, the book relies on the assumption that superintelligent AI is likely to emerge through a discontinuous jump in the capabilities of an AI system, rather than through gradual progress. This assumption is crucial to the argument that a single AI system could gain a “decisive strategic advantage”, that the alignment problem cannot be solved through trial and error, and that there is likely to be a “treacherous turn”. Hence, the discontinuity assumption underlies the book’s conclusion that existential catastrophe is a likely outcome.

The argument in Superintelligence combines three features: (i) a focus on the alignment problem, (ii) the discontinuity assumption, and (iii) the resulting conclusion that an existential catastrophe is likely.

Arguments that abandon some of these features have recently become prominent. They also generally tend to have been made in less detail than the early arguments.

One line of argument, promoted by Paul Christiano and Katja Grace, drops the discontinuity assumption, but continues to view the alignment problem as the source of AI risk. Even under more gradual scenarios, they argue that, unless we solve the alignment problem before advanced AIs are widely deployed in the economy, these AIs will cause human values to eventually fade from prominence. They appear to be agonistic about whether these harms would warrant the label “existential risk”.

Moreover, others have proposed AI risks that are unrelated to the alignment problem. I discuss three of these: (i) the risk that AI might be misused, (ii) that it could make war between great powers more likely, and (iii) that it might lead to value erosion from competition. These arguments don’t crucially rely on a discontinuity, and the risks are rarely existential in scale.

It’s not always clear which of the arguments actually motivates members of the beneficial AI community. It would be useful to clarify which of these arguments (or yet other arguments) are crucial for which people. This could help with evaluating the strength of the case for prioritising AI, deciding which strategies to pursue within AI, and avoiding costly misunderstanding with sympathetic outsiders or sceptics.


What should rationalists think about the recent claims that air force pilots observed UFOs?

28 мая, 2019 - 01:02
Published on May 27, 2019 10:02 PM UTC

NY times has a new material about possible observations of UFOs by air force pilots. Should rationalists use their arsenal of new ideas about how the world works – simulation, superintelligent AIs etc – to explain these things, or should they be of more sceptical side and explain them via some combination of biases and hoaxes?

For example, one could speculate that an alien AI have terminated long before now, but some of its self-replicating robots colonised the universe. Or may be we live in a simulation which is full of glitches and viruses?

It is possible to create many hypothesis like these with infinite explanatory powers.


Was CFAR always intended to be a distinct organization from MIRI?

27 мая, 2019 - 19:58
Published on May 27, 2019 4:58 PM UTC

When what would later become CFAR first started as a series of rationality workshops run under the purview of MIRI in 2012, was it always the intention of CFAR's founders to have it be a project/organization distinct from MIRI since its beginning? Or, did CFAR's founders decide to incorporate CFAR as an organization separate from MIRI after it became clear the rationality workshops weren't a fit project for MIRI itself later on?


Von Neumann’s critique of automata theory and logic in computer science

26 мая, 2019 - 07:14
Published on May 26, 2019 4:14 AM UTC

Quote from The General and Logical Theory of Automata. Corrected some typos using this version. H/T Hacker News.

There exists today a very elaborate system of formal logic, and, specifically, of logic as applied to mathematics. This is a discipline with many good sides, but also with certain serious weaknesses. This is not the occasion to enlarge upon the good sides, which I have certainly no intention to belittle. About the inadequacies, however, this may be said: Everybody who has worked in formal logic will confirm that it is one of the technically most refractory parts of mathematics. The reason for this is that it deals with rigid, all-or-none concepts, and has very little contact with the continuous concept of the real or of the complex number, that is, with mathematical analysis. Yet analysis is the technically most successful and best-elaborated part of mathematics. Thus formal logic is, by the nature of its approach, cut off from the best cultivated portions of mathematics, and forced onto the most difficult part of the mathematical terrain, into combinatorics.The theory of automata, of the digital, all-or-none type, as discussed up to now, is certainly a chapter in formal logic. It would, therefore, seem that it will have to share this unattractive property of formal logic. It will have to be, from the mathematical point of view, combinatorial rather than analytical.Probable Characteristics of Such a Theory. Now it seems to me that this will in fact not be the case. In studying the functioning of automata, it is clearly necessary to pay attention to a circumstance which has never before made its appearance in formal logic. Throughout all modern logic, the only thing that is important is whether a result can be achieved in a finite number of elementary steps or not. The size of the number of steps which are required, on the other hand, is hardly ever a concern of formal logic. Any finite sequence of correct steps is, as a matter of principle, as good as any other. It is a matter of no consequence whether the number is small or large, or even so large that it couldn’t possibly be carried out in a lifetime, or in the presumptive lifetime of the stellar universe as we know it. In dealing with automata, this statement must he significantly modified. In the case of an automaton the thing which matters is not only whether it can reach a certain result in a finite number of steps at all but also how many such steps are needed. There are two reasons. First, automata are constructed in order to reach certain results in certain pre-assigned durations, or at least in pre-assigned orders of magnitude of duration. Second, the componentry employed has in every individual operation a small but nevertheless non-zero probability of failing. In a sufficiently long chain of operations the cumulative effect of these individual probabilities of failure may (if unchecked) reach the order of magnitude of unity-at which point it produces, in effect, complete unreliability. The probability levels which are involved here are very low, but still not too far removed from the domain of ordinary technological experience. It is not difficult to estimate that a high-speed computing machine, dealing with a typical problem, may have to perform as much as 10^12 individual operations. The probability of error on an individual operation which can be tolerated must, therefore, be small compared to 10^-12. I might mention that an electromechanical relay (a telephone relay) is at present considered acceptable if its probability of failure on an individual operation is of the order 10^-8. It is considered excellent if this order of probability is 10^-9 Thus the reliabilities required in a high-speed computing machine are higher, but not prohibitively higher, than those that constitute sound practice in certain existing industrial fields. The actually obtainable reliabilities are, however, not likely to leave a very wide margin against the minimum requirements just mentioned. An exhaustive study and a nontrivial theory will, therefore, certainly be called for. Thus the logic of automata will differ from the present system of formal logic in two relevant respects. 1. The actual length of “chains of reasoning,” that is, of the chains of operations, will have to be considered. 2. The operations of logic (syllogisms, conjunctions, disjunctions, negations, etc., that is, in the terminology that is customary for automata, various forms of gating, coincidence, anti-coincidence, blocking, etc., actions) will all have to be treated by procedures which allow exceptions ( malfunctions ) with low but non-zero probabilities. All of this will lead to theories which are much less rigidly of an all-or-none nature than past and present formal logic. They will be of a much less combinatorial, and much more analytical, character.


Micro feedback loops and learning

26 мая, 2019 - 03:50
Published on May 26, 2019 12:50 AM UTC

Tldr: some not-particularly-ordered thoughts about how learning works in humans, via the example of singing.

A high-tech feedback loop

I recently discovered Singer’sStudio, an iPhone app for voice training that is approximately the Best Thing Ever (h/t Raemon). It’s a work of pedagogical art, and describing what it does right pulls together various nebulous thoughts I’ve had about learning and developing intuitions for a particular domain.

How the app works: it gives me various singing exercises, and then tells me in real-time, via a pretty graph, if I’m singing on pitch. That’s it.

Okay, it does have a few other features, like:

  • A total score and breakdown by % of notes correct after each exercise.
  • Letting me play myself back to see what being on vs slightly off pitch sounds like.
  • Helpful text prompts such as “keep your tongue behind your teeth.”
  • A built-in progression from easier to more complicated exercises.
  • Exercises for a specific skill, like switching from head to chest voice.
  • The ability to add challenge by turning down/off the piano playing my notes as I sing them.
  • Not a built-in part of the app, but I can glance away from the screen and sing without the visual feedback, and then immediately check how I did.

But I’m pretty confident that the pitch contour is the most important part, and an app that only did that thing would be almost as good on the skill-building side.

There are obviously a lot of singing sub-skills other than hitting notes, like vocal timbre, head versus chest voice, breathing from the diaphragm, enunciating consonants clearly, and singing vowels that don’t sound weird. A voice teacher would be able to comment on these directly.

Still, in practice the app ends up causing me to improve on a bunch of these sub-skills as well, to the extent that getting them wrong results in singing off-pitch. The instantaneous feedback is doing a lot of work here – if the app only gave me a score at the end of each exercise, I predict I would end up quite stuck on what to change up in order to do better. However, I’ve ended up e.g. remembering to breathe from my diaphragm, because not doing that reliably resulted in my voice being wobbly, which was obvious on my pitch contour. (It also has instructions for breathing exercises as part of each warm-up, but I only actually did those exercises once or twice; they are boring compared to singing and getting pretty graphs.)

It’s amazing how fast my brain learned to turn a smooth, stair-like pitch contour into dopamine hits, and feel a flicker of pain every time I jumped above or below the note before finding it. I end up very motivated to play around with any variables that make the graph prettier.


It occurs to me that another feature, which I think is less key than the instantaneous pitch contour but still pretty important, is how it slightly gamifies the entire thing, and thus makes it addictive. There are % scores at the end of the exercise! And points! It logs my all-time high score for each exercise so I can try to beat it! It also logs how many minutes a day I’ve practiced. All of this makes me more likely to use it, and actually putting in the time is a key part of training any skill.

It’s also cool that I can listen to my voice and, in addition to catching mistakes (ouch!), notice when hey, wow, I actually sounded good there. This lets me gradually figure out what correlates with liking how my voice sounds, and it also gives me a warm glow of satisfaction and helps me feel like a Real Singer.

(I’m not sure if the app is cheating by doing some kind of post-processing on these recordings to make them sound pretty. Normally I hate recordings of myself speaking, let alone singing. Still, I’ll take it.)

It also seems relevant that there’s zero embarrassment factor – this isn’t a human watching me and judging me for daring to sing when I kind of suck. I’m pretty shameless, as humans go, but I’ve been slightly nervous with every voice teacher I’ve had – they’re an expert! they’re probably really unimpressed! – and tension is not good for singing well. With this, I can sing my heart out in the privacy of my apartment, and even feel safe experimenting.

How does this generalize?

This is a post about singing, but it’s also a post about learning skills in general.

Learning to sing (or to play the piano, tie one’s shoes, draw, dance, swim; all the things commonly known as procedural memory) isn’t like memorizing a list of dates for a history test. There are some steps that can usefully happen in the explicit verbal loop, like “remember to breathe from the diaphragm”, but the end goal is that basically nothing is being held in working memory, and everything happens on the level of microsecond-to-second intuitions and muscle memory.

I think there are a really large number of skills, physical and mental, that are somewhere on a gradient between this and explicit memorization. Chess seems like a good example; it’s possible to play it from knowledge of the rules and explicit reasoning/strategizing, and there’s always some of this, but analyzing the entire game explicitly is intractable, and from what I’ve heard, experts heavily rely on intuition. ICU nursing and running events, both things I have more personal inside knowledge of, are maybe a bit closer to the intuitive or procedural end; the time pressure and urgency, and the sheer amount of stuff going on, makes explicitly reasoning through each decision less useful. Something like accounting is further towards the explicit end, but even that gets a lot more automatic with practice, and intuition is what makes something jump out as “wrong” to me.

Practicing a procedural skill without the benefit of a clever app involves two steps – trying it out (controlling my vocal chords to sing a set of notes), and being able to evaluate how good or bad a given attempt was (my “ear”). Becoming an expert involves improving both of these functions; at any given moment, in order for an instance of practicing to lead in the direction of progress, the evaluation process needs to be more accurate than the execution process.

I’ve heard anecdotes about the phenomenon of having discernment for “good X” that far exceeds one’s ability to perform X, and I’ve occasionally experienced this for other things, like writing. Therein lies frustration and embarrassment and self-judgement. In my experience, this often causes people to avoid practicing a skill, even though really it’s a good thing if your taste for X is refined enough to notice your own mistakes – that's how you know what to do differently the next time!

With singing, past a certain level I struggle to improve my ear enough to notice mistakes, which limits how much progress I can make just singing by myself at home. Voice teachers and other sources of feedback can help, but this is slower and thus less useful for shaping, in the same way that giving a dog a treat with a 5-min delay is going to be much less effective for training purposes.

This app creates a feedback loop on the sub-second level, using a different channel (my visual cortex) that doesn’t interfere with singing or listening, and which is a lot more accurate than my own internal sense for whether I’m on pitch. My hypothesis is that some lucky people do have this sort of high-quality discernment for pitch already developed in their brains, or develop it young, and they’re the ones who learn how to sing without much effort – and, because singing badly is tied up in embarrassment and shame, they’re probably the people who tend to become excellent singers at all. I expect most people with my level of innate talent, or lack thereof, just give up.

The app also lets me test and calibrate my sense of pitch more directly, so that maybe, someday, I’ll be able to tell in real-time, on my own, if I’m landing the notes in a song. (Currently I cheat and use VocalPitchMonitor for this, since SingerStudio isn't quite cool enough to let me upload random sheet music and use that to get feedback on real songs.) I play myself back and stare at the pitch contour, and try to hear the slight wrongness when the line indicates I’m just barely sharp or flat; it’s better than previous “ear training” I’ve done with a teacher or out of a book, and I’m hopeful.


This app is a really cool category of thing, that’s only possible at all due to fairly recent technological advances, and there are probably a ton more instances that I don’t know about.

I’m curious where else this has been explored. Singing may be an easy case, because measuring a single straightforward variable, pitch, gets you so far. I can imagine an app that trains, say, krav maga fighting techniques, via video analysis and/or accelerometer data, but I’m not sure that’s possible yet given current tech.

It has me thinking about other pedagogical techniques, though. Martial arts teachers will shout real-time feedback at you ("turn your hips more! get your knee higher!") I’ve taught swimming, and one issue is that waiting until a swimmer finishes a lap before giving any feedback introduces a huge delay, but grabbing onto them every time they do something slightly wrong is incredibly irritating and disruptive. Now I’m imagining giving them waterproof headphones and narrating the feedback in real time (“elbow higher please”, “roll your shoulder deeper into the water”, “keep your head back when you breathe”, etc etc.) This would be so cool.

My friend, when I brought this up, recommended I look up Tagteach. It seems to be largely based on clicker training, but lays out a bunch of thoughts I’ve had on pedagogy, like the importance of breaking a skill down into really small increments where success is easily measurable. Their site claims this protocol has been used in dance and sports coaching, but also business skills training and medical school; I’d be curious to know what this actually means in practice, but it does hint that it could be useful beyond purely physical skills (though presumably it requires success to be easily visible to the trainer; clicker-training accounting, or anything where a lot of the process is mental, seems hard!)

Of course, one of the awesome things about my app is that it removes the need for costly one-on-one time with an instructor. For swimming, could an accelerometer measure forward speed and narrate that? Speed relies on getting an absurdly large number of muscle movements just right, but the same thing is true of singing on pitch, and the single-input feedback seems to be enough to guide me towards progress. As a bonus, it could play generic reminders and prompts (I’m assuming the Singer’sStudio prompts aren’t responsive to specific mistakes I make, but they’re still useful.)

It also occurs to me that if getting direct and immediate feedback on your evaluation process is key to improving it, it could be useful to focus on that directly, separately from executing the skill itself – if you’re better at catching your own mistakes, later practice will be more valuable. I’m imagining a gymnast watching videos of Olympic gymnastics that have real-time commentary analyzing what the athletes are doing, noting successes and mistakes.

I would be curious to hear any examples others have of this.


Say Wrong Things

25 мая, 2019 - 01:11
Published on May 24, 2019 10:11 PM UTC

There are many ways you might approach being less wrong.

A popular one is to make fewer wrong statements; to say fewer wrong things.

Naively it would seem this is a recipe for success, since you just say more things that are true and right and fewer things that are false and wrong. But if Goodhart has anything to say about it, and he does, you'll find ways to maximize the measure at the expense of the original objective.

Assuming the real objective is something like "have a more complete, precise, and accurate model of the world that better predicts the outcome of subjectively unknown events", then we can quickly see the many ways Goodharting can lead us astray if we focus too much on appearing less wrong. We might:

  • make fewer claims than we could, pulling us away from completeness even as we appear less wrong;
  • make weaker claims than we could, pulling us away from precision;
  • and, a perennial favorite, filter the claims we publicly make so we appear less wrong than we really are by hiding our least confident claims.

The first two can be corrected with better calibration, that is by making statements with confidence intervals or likelihoods that proportionally match the observed frequency of correctness of similarly confident claims. But simply suggesting someone "be better calibrated" is not a motion they can make; it's an outcome of taking actions towards increasing calibration. As good a place to start as any for improving calibration is the forecasting literature, if that's what you'd like to do.

The third is more tricky, though, because it's less directly about claims being made and their probability of correctness and more about social dynamics and how you appear to other people. And for that reason it's what I want to focus on here.

Appearing Less Wrong

I've met a lot of people in my life who are experts at not looking as stupid as they are.

That's kind of harsh. Maybe a nicer way to say it is that they are experts at appearing to be better at making correct predictions about the world than they actually are.

Some of their techniques are just normal social tricks: projecting confidence, using social status, the ever-abused term "gaslighting", and other methods of getting people to believe they are right even when a more careful examination would reveal them to be mistaken. These are people we all love to hate and love when we can call them on their bullshit: overconfident academics, inflated politicians, self-important internet intellectuals, and those people whose idea of social interaction is to say "well, actually...".

But there's a way to avoid looking stupid that is more pernicious, less amenable to calling out, and that subtly drags you towards local maxima that trap you mountains and valleys away from more complete understanding. And it's to shut up and not tell anyone about your low confidence beliefs.

It is extremely tempting to do this. Among the many benefits of keeping low probability claims to yourself:

  • you have a high accuracy ratio of publicly made claims, making you look right more often when observed;
  • you say only things that, even when wrong, turn out to be wrong in conservative ways that still make you look smart;
  • and you accrue a reputation of being right, usually conferring social status, which can feel really good.

The only trouble is that this approach is too conservative, too modest. It's easy to justify this kind of outward modesty as keeping up appearances in a way that is instrumental to some bigger goal, and you say to yourself "I'll still make low probability claims; I'll just keep them to myself", but down that path lies shadow rationality via compartmentalization. You can try it, but good luck, because it's a dark art that hopes to do what human brains cannot, or at least cannot without some sufficiently powerful magic, and that magic traditionally comes with vows not to do it.

Meanwhile, out in the light, finding models that are better predictive of reality sometimes requires holding beliefs that appear unlikely to be true but then turn out to be right, sometimes spectacularly so, although semper caveat, all models are wrong, but some are useful. And then you have to go all in sometimes, exploring the possibility that your 10% guess turns out to be 100% correct, minus epsilon, because if you don't do this you'll do no better than the medieval Scholastic holding to Aristotelian physics or the early 20th century geologist ignoring the evidence for continental drift, forever locked away from taking the big risks necessary to find better, more accurate, precise, and complete models.

Okay, so let's say you are convinced not to try so hard to appear more right than you are. How do you do that?

Say It Wrong

So I suppose it's nothing so much harder than just telling people your claims, even if you have low confidence in them, and seeing how they react, although depending on the circumstances you'll probably want to adequately explain your confidence level so they can update on it appropriately. The trouble is getting yourself to do that.

I can't change your mind for you, although thankfully some folks have developed some techniques that might help if you're not interested in over-solving that problem. What I can do is point out a few things that might help you see where you are being too modest, nudge you towards less modesty, and create an environment where it's safe to be less modest.

  • Look for the feeling of "pulling your punches" when you are telling people your ideas.
  • Ramble more and filter less.
  • Worry less about how you look to others.
  • Relatedly, increase your ability to generate your own esteem so you less need to receive it from others, giving you more freedom to make mistakes.
  • Encourage others to tell you their half-baked ideas.
  • When they do, be supportive.
  • Take a collaborative, nurturing approach to truth seeking.
  • Play party games like "what do you think is true that you think no one else here also thinks is true?" and "what's your least popular opinion?".
  • And my personal favorite, write up and share your immodest, lower-confidence ideas that you think deserve exploration because they have high expected value if they turn out to be right.

I suspect that the real reason most people try too hard to appear more right than they are is fear—fear of being wrong, fear of looking stupid, fear of losing status, fear of losing prestige, fear of losing face, fear of being ostracized, fear of being ignored, fear of feeling bad, fear of feeling lesser. I see this fear, and I honor it, but it must be overcome if one wishes to become stronger. And when you fear being more wrong, you will be too careful to ever become as less wrong as you could.

To sum it all up pithily:

To become less wrong, you must give up being most right.