Concerning Superintelligence

These are my recommendations of key texts to read  if you really want to get familiar with   Superintelligence. 

SI-1. Good, I. J. (1966). Speculations concerning the first ultraintelligent machine. In Advances in computers (Vol. 6, pp. 31-88). Elsevier.

Irving John (Jack) Good was mathematician who worked with Alan Turing and made significant contribution to braking the Enigma codes. One could regard him as Turing’s statistician. Good later worked with British AI pioneer and computer designer Donald Michie. Good devoted much of his later life to research in Bayesian statistics. Goods paper cited above was the first to clearly spell out ultraintelligent machines and can be rightly viewed as the basis of the superintelligence discipline today. This paper stated:

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control

This short paragraph not only presages the idea of superintelligent AI, it also laid the groundwork for subsequent Paperclip Apocalypse scenarios and the drive for AI safety considerations. Good was particularly a credible messenger due to his early intimate and highly knowledgeable technical familiarity and experience with highly complex and capable computers.

SI-2. Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. New York: Oxford University Press.

Bostrom’s book was much waited by the superintelligence (SI) community, and in some respects provided the academic sanctioning of runaway-AI potential for harm, and AI-safety, as legitimate scholarly topics for discussion. In some ways the runaway SI apocalypse scenarios act to counterbalance Ray Kurzweil’s Exponentiality of all things technological and Singularity visions.

SI-3. Drexler, K.E. (2019): Reframing Superintelligence: Comprehensive AI Services as General Intelligence, Technical Report #2019-1, Future of Humanity Institute, University of Oxford

This is a must read by Eric Drexler, pioneer of nanotechnology . This report projects a possible, if not likely, trajectory of AI development that envisions emergence of asymptotically comprehensive, superintelligent-level AI services. Drexler has been prescient regarding the importance of and trajectory of nanotechnology.

SI-4.Yampolskiy, R. V. (2015). Artificial Superintelligence: a futuristic approach. CRC Press.

While maintaining a focus on AI and superintelligence safety, Roman Yampolskiy brings additional dimensions to discussions of superintelligence. I am not quite sure why the need to use the term Artificial in the title and the discussion. Superintelligence is not now and will never be a normal or natural attribute; I view adding artificial to superintelligence as redundant.

The book includes interesting and useful discussions on topics such as AI-Completeness and AI-Hardness, Mind Design and associated taxonomies of real and speculative mind design space. Most of the intensity and depth of discussion though is focused on the harm that SI can bring (and according to the author and many of the references cited, viewed as very likely to occur.) The detailed references provided are exceptional. Personally, I would prefer to see more discussion of the positive aspects of SI and the hard problems it can and should solve first.

SI-5. Philip Larrey (2017), Would Super-Human Machine Intelligence Really Be Super-Human? in G. Dodig-Crnkovic and R. Giovagnoli (eds.), Representation and Reality in Humans, Other Living Organisms and Intelligent Machines , (Studies in Applied Philosophy, Epistemology and Rational Ethics 28, DOI 10.1007/978-3-319-43784-2_19)


AI Recommended Readings / Possible. Minds

added a section for Artificial / computational / machine intelligence recommended readings on the main site — here.

noted also that  “We’re also paying attention to the applicability of AI/MI concepts to the space of What Sloman calls ‘possible minds’. “The idea is that the space of possible minds encompasses not only the biological minds that have arisen on this earth, but also extraterrestrial intelligence, and whatever forms of biological or evolved intelligence are possible but have never occurred, and artificial intelligence in the whole range of possible ways we might build AI”

The initial list of principal textbooks include:
1. Russell, S. J., & Norvig, P. (2016). Artificial intelligence: a modern approach.
This is an updated edition of the 2010 version containing extensive current references.
[note the book is getting hard to find sometimes due to demand, and its being the definitive AI textbook, check the edition you are using/getting]

2. Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT Press. This is an updated (2nd) edition of the 1998 version.
In addition to his University of Alberta academic appointment, Richard Sutton is now the head of Alphabet/Google DeepMind Alberta operations.

3. Nilsson, N. J., & Nilsson, N. J. (1998). Artificial intelligence: a new synthesis. Morgan Kaufmann.

4. Poole, D. L., Mackworth, A. K., & Goebel, R. (1998). Computational intelligence: a logical approach (Vol. 1). New York: Oxford University Press.

5.  Artificial Intelligence: Foundations of Computational Agents 2nd Edition by the same authors.


Stanley & Lehman – Why Greatness Cannot Be Planned

Fascinating insights by Computer Science / Artificial Intelligence profs …

some have summarized their insights by writing: “only by doing activities that fulfill our curiosity without any pre-defined objectives, true creativity can be unleashed. They call this the ‘Myth of the Objective’: Objectives are well and good when they are sufficiently modest … In fact, objectives actually become obstacles towards more exciting achievements, like those involving discovery, creativity, invention, or innovation—or even achieving true happiness… the truest path to “blue sky” discovery or to fulfill boundless ambition, is to have no objective at all.”

some of Stanley’s and Lehmans insights:


  • “The flash of insight is seeing the bridge to the next stepping stone by building from the old ones. ”


  • “[Picbreeder] is just one example of a fascinating class of phenomena that we might call non-objective search processes, or perhaps stepping stone collectors. The prolific creativity of these kinds of processes is difficult to overstate”


  • “ measuring success against the objective is likely to lead you on the wrong path in all sorts of situations”


  • “You can’t evolve intelligence in a Petri dish based on measuring intelligence. You can’t build a computer simply through determination and intellect—you need the stepping stones. ”


  • “ambitious objectives are the interesting ones, and the idea that the best way to achieve them is by ignoring them flies in the face of common intuition and conventional wisdom. More deeply it suggests that something is wrong at the heart of search. ”



I find their books inspiring and insightful. Reframing questions and providing different lines of attack on AI and Search Optimization to Ambitious Goals …





Demis Hassabis talk on General Artificial Intelligence

Demis Hassabis: Towards General Artificial Intelligence – talk at Center for Brains, Minds and Machines (CBMM). [Background: r. Demis Hassabis is the Co-Founder and CEO of DeepMind, the world’s leading General Artificial Intelligence (AI) company, which was acquired by Google in 2014 in their largest ever European acquisition.

The talk  draws  on Demis’  eclectic experiences as an AI researcher, neuroscientist and video games designer.

see also.

deep learning drizzle

u/kmario23 over at reddit points to a wonderful new resource m the deep learning drizzle. [on Github]

I have collected a list of freely available courses on Machine Learning, Deep Learning, Reinforcement Learning, Natural Language Processing, Computer Vision, Probabilistic Graphical Models, Machine Learning Fundamentals, and Deep Learning boot camps or summer schools.

So I checked it and immediately got involved watching Ian Goodfellow …

Ian and his advisor wrote this book …. take a look at it.

Goodfellow posted pdfs of his talks here  is worth following

AlphaGo Documentary 2017

the  1 1/2 movie about AlphaGo, Deep Mind’s AI program that beat the best GO player in the world. AMAZING .  MUST WATCH if you are really into real AI.

If you want to really understand what happened and how, and how Deepmind really works … you’ve got to watch the  documentary.

Lots of information about the game of GO … a great way to bring 8000 years of GO history and the future together