(Michael Chinen)

Like everyone else, I’ve been playing with ChatGPT for many different tasks. Here is a short story that I used ChatGPT to create. It seems able to do the basic request, which was to use mysticism to tie longevity biotech progress and mycology together with a metaphor in a vaguely sci-fi style.  I decided the theme while writing, and asked it to write and rewrite sections with my direction, and I hand picked which paragraphs to use for the final result which I’m sharing below.  The full sequence of prompts and responses is included at the end, along with some discussion.

The story is reasonable, and doesn’t have any obvious mistakes at the sentence level. I was impressed that it kept global structural integrity as well as it did, given the length of the story (~2900 words). Generally, I thought the story and quality of writing was better than something I could come up with in an hour. It starts half of the paragraphs with ‘As I’; it has clear storytelling flaws; it doesn’t always respect literary devices. For example it gives up metaphors by exploring metaphysics on the story too easily. It did, however, use apt metaphors at other times: how nature represented by the forest can be both powerful, beautiful, scary, and if we look closely at it we can find solutions. On the other hand, I’m not a creative writer, and perhaps this story is of much lower quality than I realize. Maybe that’s the sweet spot: better than a total newbie, but not an expert. A lot of this task was about maintaining a certain style, and from the way the GPT-family works, it’s reasonable to think it could do this well.

At the same time, it misinterpreted or failed to acknowledge several instructions. I had to ask it to rewrite sections. Still, it was an interesting process that had me glued to the screen for an hour. Other ChatGPT tasks such as ‘help me research subject x’, ‘write code with me’ or ‘be my chat companion’ were less interesting, and also felt less powerful, and had more obvious mistakes (example: when I was researching standing desks, the agent was certain that standing desks were used by having people stand on top of the desk surface). Anything that requires expertise seems a risky thing to ask of ChatGPT.

One immediate application this gives us today is the ability for people that aren’t so actively doing media creation to try it out and get an appreciation of the feeling for writing, as opposed to actually using it to publish stories. If you want to publish it, there’s also a reduced sense of the typical apprehension of releasing a part of something you’ve worked on into the world because you can always blame it on ChatGPT. I, personally woudn’t feel comfortable sharing this story otherwise.

Depending on the medium, it might not even be that bad for publishing. I think novels and essays are usually quite refined and may be too high a bar to challenge at this point, but someone could make the argument if these large language models are used for assisting in creative endeavors with lower requirements, such as some Netflix series or movies. Netflix movies like Bright already feel like they are the result of taking current social trends and topics as input to some machine learning algorithm that spits out a script, and are often quite bad. Although here, the problem may not be so simple. In Bullshit Jobs, David Graeber mentioned an inside story from recent media producers explaining why so many films and TV shows are mediocre. It had to do with so many unnecessary ‘creative executives’ that don’t do anything for the script, except make small changes to be able to claim they did something. I have no idea how true that is, but this is more of a societal issue. But I could totally see passing the scripts through ChatGPT and getting better stories than some of the current ones.

I have many more ChatGPT interactions, and want to share them, but since I’ve already posted on it I’ll give it some time to see if that feeling survives the hype.

Complete prompt/response:

I’d like to write a scifi story with you.  But first I want to know your limitations.  Since I’ll be asking you to edit and rewrite sentences, how many words or sentences or chapters can I expect you to be able to remember?  This will be important for keeping the story consistent and focused.

ChatGPT: As a large language model trained by OpenAI, I am not capable of remembering specific sentences or chapters of a story. My purpose is to assist with generating text based on the input and prompts provided to me. I can help with editing and rewriting sentences to improve the clarity and consistency of the story, but I don’t have the ability to remember and keep track of specific details across multiple interactions. My main goal is to help you with your writing and to provide suggestions and ideas based on the information you give me.

Me: I will only ask you to remember my text input from this conversation, but I still need to know how many tokens you can remember.

ChatGPT: As a language model, I don’t have the ability to remember specific sentences or pieces of text. My primary function is to generate text based on the input and prompts provided to me. I can provide suggestions and ideas to help with your writing, but I don’t have the ability to remember and keep track of specific details across multiple interactions. My main goal is to assist with the writing process and to help improve the clarity and consistency of the text.

Me: For example, if I give you an entire chapter of say, 2000 words, can you keep that all in memory, or will it be problematic?

ChatGPT: As a language model, I don’t have the ability to remember specific sentences or pieces of text, including entire chapters of a story. My primary function is to generate text based on the input and prompts provided to me. I can provide suggestions and ideas to help with your writing, but I don’t have the ability to remember and keep track of specific details across multiple interactions. My main goal is to assist with the writing process and to help improve the clarity and consistency of the text. If you have a specific chapter that you would like me to help with, you can provide the text to me and I can assist with editing and rewriting sentences to improve the flow and coherence of the chapter. However, I won’t be able to remember the chapter in its entirety across multiple interactions.

Me: Ok fuck it, let’s go.  I hope swearing is okay.

ChatGPT: As a language model, I am a neutral AI assistant and don’t have personal preferences or opinions on language. My primary goal is to assist with your writing and to provide suggestions and ideas to help improve the clarity and coherence of the text. Whether or not to use swear words in your story is entirely up to you as the author. My role is to help with the writing process and to provide feedback and suggestions based on the information and prompts you provide to me.

Me: For this short story, the first word of every sentence will combine to be meaningful comentary on this story.  After this input sequence, I will begin writing the story, and you can edit or continue the story in your response.  If I have requests or commands, {I’ll use curly brackets to indicate this as separate from the story}.  You can also use those curly brackets to give me feedback or directions.   Do you think this is a good system to collaborate, or would you advise some modification to it?

ChatGPT: This is a creative and unique way to approach collaborative writing. Using the first word of each sentence to create a separate commentary on the story is an interesting concept, and I’m looking forward to seeing how it will develop in the story. One potential issue with this approach is that it may be difficult to maintain a consistent and coherent narrative while also crafting the commentary using the first words of each sentence. It may be helpful to plan out the overall structure and direction of the story before beginning to write, and to consider how the commentary will fit in with the narrative. Using the curly brackets to indicate requests and commands is a good way to keep track of these, and I will be sure to follow any instructions provided in this way. Overall, I think this is a promising approach to collaborative writing, and I’m excited to see what you come up with. Let me know if you have any specific requests or ideas for the story, and I will do my best to assist.

Me: When I woke up this morning, I heard the rain hitting the window and the roof.  It’s the only sound I’m used to.

ChatGPT: I’m not sure what I was expecting to see when I got out of bed and made my way to the kitchen, but the sight of my empty fridge and the dirty dishes piled up in the sink was almost a relief. At least it was familiar. As I poured myself a cup of coffee and looked out the window, I couldn’t help but wonder what my life would be like if I had made different choices. But it’s too late for that now. All I can do is try to make the best of what I have.

Me: {rewrite that paragraph respecting the rule that the first word of sentences is its own commentary and should be grammatically correct}

ChatGPT: When I woke up this morning, I heard the rain hitting the window and the roof. It’s the only sound I’m used to. Although I didn’t know what to expect when I got out of bed and made my way to the kitchen, the sight of my empty fridge and the dirty dishes piled up in the sink was almost a relief. At least it was familiar. As I poured myself a cup of coffee and looked out the window, I couldn’t help but wonder what my life would be like if I had made different choices. But it’s too late for that now. All I can do is try to make the best of what I have.

Me: {add a hint of murakamian mysticism from something outside the window}

ChatGPT: When I woke up this morning, I heard the rain hitting the window and the roof. It’s the only sound I’m used to. As I made my way to the kitchen, I couldn’t help but notice the strange light coming from outside the window. It was as if the rain was glowing, casting an otherworldly glow on the street below. I poured myself a cup of coffee and stepped outside onto the balcony, where I was greeted by the sight of a fox standing in the rain, staring up at me with piercing eyes. In that moment, I felt a strange sense of connection to something beyond my own world, as if the fox was trying to communicate something to me.

Me: I would have guessed that foxes don’t care much for the rain.  But then again, the stone-moss petrichor that the light wind pushed around suggested it hadn’t been raining for so long.  

ChatGPT: As I stood on the balcony, watching the fox in the rain, I realized that I had been wrong about the fox’s reaction to the weather. The fox didn’t seem to mind the rain at all, and instead seemed to be enjoying the fresh scent of the stone-moss petrichor that the light wind was blowing around. It occurred to me that it must have been a while since it last rained, and the fox was taking advantage of the moment to explore and enjoy the world around it. I couldn’t help but feel a sense of curiosity and wonder at the fox’s behavior, and I found myself wanting to know more about this mysterious creature.

Me: The fox was at a distance far enough that it wouldn’t feel threatened by anything I did.  When I’m out in the woods, sometimes I make a loud hoot to let the animals know I am there.  It’s entirely possible this fox and I had communicated in the past.

ChatGPT: As I watched the fox from the balcony, I wondered if it had ever noticed me before. The fox was far enough away that it didn’t seem to be threatened by my presence, and I couldn’t help but wonder if it had somehow sensed me even before I stepped outside. I remembered the times when I had been out in the woods and made a loud hoot to let the animals know I was there. It was entirely possible that this fox and I had communicated in this way before, and that it was responding to my presence now. The thought of this possibility filled me with a sense of excitement and curiosity, and I found myself wanting to know more about the fox and its behavior.

Me: I see that you are treating my text as if it’s guidance and not the actual story.  That’s fine, let’s continue like this.  Add a faint and looming sense of danger in the next three paragraphs.

ChatGPT: As I continued to watch the fox from the balcony, I couldn’t shake the feeling that there was something off about the situation. Despite the peacefulness of the rain and the fox’s calm demeanor, I couldn’t shake the sense of unease that was growing inside me. It was as if something was lurking just out of sight, waiting to reveal itself at any moment. I felt my heart rate increase and my palms start to sweat, and I couldn’t help but wonder what was causing this sudden sense of danger.

I tried to tell myself that I was just being paranoid, but the feeling only grew stronger. The fox seemed to sense it too, and it turned its head to look in the direction of the woods behind my apartment building. I followed its gaze, but I couldn’t see anything out of the ordinary. Still, the sense of danger was palpable, and I couldn’t shake the feeling that something was about to happen.

I considered going back inside and locking the door, but something held me back. I was drawn to the fox, and I felt a strange sense of connection to it. As I watched it, I realized that it was the only thing keeping me grounded in the face of the looming danger. I took a deep breath and tried to steady my nerves, determined to face whatever was coming my way.

Me: I felt existential dread from the sudden fear that this world would cease to exist if I went back inside the apartment.  Or worse, that the entire world consisted entirely of what was in front of me.

ChatGPT: As I stood on the balcony, facing the fox and the sense of danger that surrounded us, I felt a sudden wave of existential dread wash over me. The fear that this world would cease to exist if I went back inside the apartment, or that the entire world consisted only of what was in front of me, was overwhelming. I felt my chest tighten and my breath quicken, and I was overcome with a sense of panic and despair.

In that moment, I realized that I had to do something to break free from this feeling. I couldn’t just stand there and let myself be consumed by fear and uncertainty. I took a deep breath and tried to focus on the fox, using it as a anchor to keep myself grounded in the face of the looming danger. Slowly but surely, the fear began to recede, and I was able to think more clearly.

I realized that I needed to take action if I wanted to survive this ordeal. I couldn’t just stand there and let the danger consume me. I had to face it head on, no matter how scared I was. With renewed determination, I took a step forward and began to move towards the woods, ready to face whatever was waiting for me.

Me: It’s quite complicated and ridiculous looking to get down to the ground from my balcony in my pajamas, but this seemed more and more logical because I couldn’t remember anything else.  

ChatGPT: As I made my way towards the woods, I couldn’t help but notice how ridiculous I must have looked, trying to navigate the difficult terrain in my pajamas. But in that moment, nothing else mattered except for my desire to face the danger and protect the fox. Despite the complications and potential embarrassment, I continued to push forward, determined to find out what was lurking in the woods.

As I neared the edge of the woods, I felt my heart pounding in my chest. I was filled with a sense of determination and fear, unsure of what I would find but knowing that I had to face it. I took a deep breath and stepped into the woods, ready to confront whatever was waiting for me.

The sense of danger was palpable as I made my way deeper into the woods. The fox followed close behind me, its presence providing a sense of comfort and reassurance. Despite the fear and uncertainty, I felt a sense of determination and strength that I hadn’t known I possessed. I was ready to face whatever was waiting for me, and I was determined to emerge from this ordeal victorious.

Me: Add a hint that the world is simulated.  This is a short story and will end within the next 20 paragraphs

ChatGPT: As I made my way deeper into the woods, I couldn’t shake the feeling that there was something strange and surreal about the world around me. The trees seemed too perfectly placed, the sky too clear and blue, and the sounds of the forest too muted and distant. It was as if I was walking through a carefully crafted simulation, and the fear and danger that I had felt earlier only added to the sense of unreality.

I tried to push these thoughts to the back of my mind, focusing instead on the task at hand. I had to find out what was lurking in the woods, and I had to protect the fox at all costs. But as I continued to move forward, the sense of unease only grew stronger. It was as if the entire world was conspiring against me, determined to keep me from uncovering the truth.

Despite these doubts and fears, I continued to push forward, drawn by a sense of curiosity and determination. I had to know what was waiting for me in the depths of the woods, and I was determined to find out, no matter the cost. As I moved closer to the center of the forest, I couldn’t shake the feeling that I was on the brink of uncovering a great secret, one that would change everything I thought I knew about the world around me.

Me: I searched for some of my dearest memories

ChatGPT: As I made my way through the woods, I found myself thinking about some of my dearest memories. I remembered the time when I was a child, playing in the park with my friends. The sun was shining, the birds were singing, and life was full of possibility. I felt a sense of nostalgia and longing, and I couldn’t help but wonder if I would ever be able to experience that kind of joy and freedom again.

I also thought about my first love, the person who had taught me what it was to truly care for someone. We had spent hours talking, laughing, and dreaming, and I had never felt more alive. But in the end, it hadn’t been enough, and we had gone our separate ways. I still thought about that person sometimes, and I couldn’t help but wonder what might have been.

As I continued to search for the source of the danger in the woods, I realized that my memories were both a source of comfort and pain. They reminded me of who I used to be, and they made me long for the things that I had lost. But at the same time, they also gave me strength and determination. They reminded me that I had faced challenges before, and that I was capable of overcoming them. And with that knowledge, I continued to push forward, determined to uncover the truth and protect the fox.

I’ve been increasingly interested in longevity for the past few years but not written much about it. A part of the reason is that I’ve tried to keep my finances out of this blog, and part of my interest in longevity is in investing. I want to talk about that in a future post. This post is about a curated selection of longevity-focused books that I’ve read.
Longevity research has been slowly but steadily accelerating since the 90’s. There has been increasing progress since the late 19th century, but until recently it has been ‘career suicide’ to dedicate one’s research to this domain.
This post has two purposes for being: I would like to capture and share my thoughts on some of the longevity books that I’ve read. Eventually, I would also like to provide a useful guide for anyone looking to start exploring longevity biotech by reading a casual non-fiction book. There are a lot of books labeled as longevity books that are really just lifestyle, diet, fitness, cosmetics, and supplement books. Those topics are tangentially related to longevity, but are not the most interesting and impactful longevity space is primarily concerned with. The authors of these books are often not even aware of the high likelihood that maximum human lifespan and healthspan will increase by significant amounts in this century. To compound the issue, some of the authors are celebrities that are about ’embracing aging’, such as Cameron Diaz’s book, and their fame means that if you search for ‘longevity book’ on Amazon you will first see one of these books.
I’ve selected four important books for longevity that are targeted to the general audience and each provide a different value. I will do mini-reviews of them here, focusing on their differential value add. The books I’m considering here are written by world class researchers that break down the research trends and explain the cellular mechanisms that will enable such a future that will slow, stop, and eventually reverse aging. In case you are new to this idea, if it sounds suspicious, perhaps because of the history of dubious claims in supplements and alternative medicine, some appeals to authority may be useful to keep you from closing the tab. The people doing the important research here include PhDs and professors at universities such as Harvard and Stanford, and publish the research in the top journals such as Nature and Cell. Institutional capital, from funds and venture capital, is backing a growing number of these companies, which requires significant diligence to vet the ideas. There are also publicly traded companies in the space, as well as highly visible private ventures such as Alphabet’s Calico and Altos Labs recent $270M round announcement, backed Jeff Bezos.

Ageless: The new science of getting older without getting old – Andrew Steele (2021)
Unlike Sinclair and Barzilai, Andrew Steele hasn’t launched any LongBio companies (or been in licensing disputes regarding products). His background is in physics. This disconnectedness actually appears to give Steele an edge in being able to provided less biased surveys of the different approaches and research. Steele does not cover every advancement and research area (e.g. fisetin is not even mentioned in the senolytics section), but Steele also manages to keep a balance between the history of research, technical discussions, and big picture trends, which makes this book very appropriate for a first foray into longevity books.

Update (Feb 2024): Andrew Steele continues to be interesting, and has a number of videos on the topic that are informative and entertaining.  I also recommend reading this blog post if you want a detailed review.

Lifespan: Why we age – and why we don’t have to – David Sinclair (2019)
This book is written with enthusiasm that is infectious at times. David Sinclair is a very high profile person in this space, having an extensive research career and having developed several businesses that have made money. While there are supporters, there are also those that express doubt on reddit that feel that his enthusiasm is a charade. I can’t comment on whether or not the book is genuine (which is not falsifiable), but I did felt that the book was well written and presents a cohesive story that benefits from Sinclair’s solid research and business background and conviction. This is certainly a field that deserves passion. Sinclair presents describes aging in terms of information theoretic entropy and accumulation of DNA damage and epigenetic noise, suggesting a focus on these DNA and cell-based mechanisms of aging, with less focus on others such as atherosclerosis. The content is naturally biased towards Sinclair’s interests describing his personal story with NAD+ and resverterol (with a little discussion of the GSK/Sirtris shutdown). It was a pleasant surprise to see the depth into which Sinclair discusses the sociological consequences of extending lifespan, beyond the typical strawman arguments. These societal questions are something that Steele and Barzilai only address at the surface level, which is acceptable, since they are not sociologists. But for the layperson reading into this field for the first time, these questions can be the gatekeepers, and it is important for a public advocate to be able to speak to them. The weakest part of the book is also the bias towards his own research, especially in the anecdotal descriptions of his and his father’s self-treatments using NMN, metformin and resveratrol, as well as to some extent the neurogenesis glaucoma work. The near miraculous effects in his father (from being near bed-ridden to being super healthy and active) are so outsized from the expected outcomes in the literature, that even with the preface that this is just n=1, this will encourage a general audience to self-experiment with similarly outsized expectations. I much rather preferred Steele’s approach, which was to say the only verified approaches are exercise and diet for now and that we just need to wait for better studies and drugs. Overall, because of Sinclair’s influence on the field and conviction, I think this book is a must-read (but should be supplemented with other perspectives).

Age Later: Health Span, Life Span, and the New Science of Longevity – Nir Barzilai (2020)
This book follows Nir Barzilai’s research and looks at centenarians in depth to see what patterns emerge. The data is naturally observational and confounded, so the trick is to come up with a theoretical causal model, and then test it in a more controlled experiment. Barzilai has access to gene banks that make this study more interesting. Barzilai talks about business and investing to a larger degree than any of the other books, including business development, fundraising, and valuations, which was refreshing to see. Barzilai also talks about metformin in considerable detail, including how he managed to design, advocate for, and launch the Targeting Aging with Metformin (TAME) trial, including getting the National Institute of Aging (NIA) to sign on. It might be natural to focus on research progress, but in practice, these interpersonal, business, and salesman ship skills are incredibly important in this field, so it is wonderful to see this book cover that in such detail. As suggested by the centenarian premise, this book does not try to look at each root cause of aging, but instead looks at individual solutions that are interesting to the author. As a result, it has coverage of different research topics when compared to the other three books (for example, a significant section on statins). As a result, I would recommend this book as a supplement to the others.

Ending Aging – Aubrey de Grey (2007)
This is a groundbreaking book that paved the way for others to follow and helped to form a movement and community/organizations (SENS). The book describes in detail seven root causes of aging and proposes theoretical solutions for each of them. Because of AdG’s background and previous book, he spends more time on the mitochondrial mechanisms. This book was written for a general audience, but it does go into enough detail that the technical sections are slower to read. This detail is refreshing at times but some of the detail will likely be lost on someone like me (e.g. little biology background) unless I go back and forth between the pages several times. The other thing is that this book does feel a little dated – it won’t contain the latest advancements, and some of the optimistic predictions of progress (which were contingent on proper funding) have already expired without being fulfilled. For these reason I would recommend the Steele or Sinclair book as a first book before approaching this one.

There are of course, many more interesting books to read. I also read Al Chalabi and Jim Mellon’s Juvenescence and Jean Hebert’s Repalcing Aging and found them more specific than the above books. Juvenescence‘s subtitle is ‘investing in the age of longevity’, but there is not that much content about actual investment and business (other than a compilation of a list of companies), which was surprising to me given Jim Mellon’s background. Rather, it spends most of the time on surveying the research progress, as well as a mixture of health advice – this part aligns closely with David Sinclair’s book, perhaps due to their business relationship. Jean Hebert’s book is a short and powerful, presenting a very different strategic approach using replacement of damage to handle the majority of the problems caused by aging, with the brain being the one thing that needs special treatment due to identity preservation. This is a thought-provoking quick read that even brings transhumanistic and philosophical questions into the mix.

All of these books are worth a read. It’s great to see so many of them coming out, especially because of the rapid pace of the research and industry. I’d say it’s important to go into these books with the right mindset. Most of the negative comments are about not being actionable feedback for people that want treatments today. Unlike the ’embracing aging’ books it doesn’t try to make you feel better about getting old, and I think this is jarring to some people as well. But the point of these books is to change the pro-aging stance that society has developed before there was anything that could be done about it. This is a field that will change life as we know it, literally, and it is just taking off now. On a parting note to get in the right mindset, I highly recommend reading Nick Bostrom’s gem The Fable of Dragon-Tyrant.

All disciplines have interesting histories that explain their development. For some reason, different studies seem to ‘value’ their history differently. Art and music students are required to study art and music history with multiple dedicated classes. Computer science and mathematics students do not have to study math or computer science history in a typical undergraduate program (if you are lucky, maybe one or two classes). Instead, if the computer science or math student is lucky, they get a charismatic professor who is a good story teller and fits in anecdotes about the creators of the topics being studied, or a reference to a book about this.

Presumably, the history of art is useful to an artist, not only for telling interesting stories to their students one day, but also to understand something deeper about where new art comes from. A deeper understanding may be useful for a number of things, including how to go about creating novel art. The creation of new abstract concepts in math and art have certain high-level similarities – they provide a new way to look at things. Perhaps the poster child for this type of thinking is Xenakis, whose existence united architecture, statistics, algorithms, and music, where knowledge across the fields had an interesting synergy. But even if we ignore cross-discipline examples, I think we will find that the innovators typically have had an interest in the history. Is the assumption for the sciences that this is correlation, and not causation? That argument can be made, but it seems less likely to me. Or is the assumption that in the interest of time, most undergraduate students don’t need to be innovators, and rather just need to understand how to solve the damn equation, not how it came to be?

Perhaps this is a straw-man. I may be generalizing from an unrepresentative experience, and it has been a number of years since I was in school. It seems like folks pursuing graduate degrees in math/computer science have more understanding of history, and because this information is easier to come by today than 20 years ago, people that self-study also pick this up naturally. In that case all I can say is that I was not aware of so much of the history involved in computer science, and I wish I had started my studies with something like James Gleick’s The Information.

Information theory is a relatively new field of the sciences. Of course, it did not spring out of nowhere. There are a few history-oriented books that describe its formation, but there is not much. Gleick’s coverage is by far the widest I’ve seen.

The book has an excellent cast of characters, starting out with long distance communication with African drums, Babbage/Lovelace and early computers, and Laplace. As the book develops the more typical founding characters in Information theory appear, with Maxwell and his heat daemons, Clausius and his entropy, Morse with his codes, Hartley, Shannon, Wiener, Turing, and Kolmogorov. What makes the book’s presentation special is the depth in which each character is gone into. There are a large amount of supporting characters and competitors that I hadn’t heard of, which provides great context for the developments. Naturally, a large amount of time is spent of the juicy rivalries such as the Shannon-Wiener relationship, but also how it fit into the rest of the world, e.g., how Kolmogorov felt about them.

I was introduced to a range of new connections that I was not aware of, including the Schrodinger (yes, that Schrodinger) connection to molecular biology and What is Life?. There were also nice teasers for the parts of info theory I haven’t had exposure to such as Quantum computing and Schorr/Feynman’s thoughts on it. There also deeper ties to fundamental math history such as the early developments in greek and arabic states from Aristotle to al-Khwarizmi. I was also unaware of the amount of obsoleted infrastructure required for telegraph networks, and the book spends a good time talking about the logistics of this kind of thing.

I very much enjoyed this book, although it still misses a few important areas. Notably, Kullback’s application of information theory to statistics, as well as Bayesian statistics and the related information criteria are not mentioned. Deep learning is also not mentioned, but the book was published in 2011, before the recent surge. Naturally, Gleick also discusses the fictional works of Borges. Unfortunately as much as I enjoy Borges, I found this to be the weakest part of the book.

At 426 pages, Gleick’s presentation is almost entirely conceptual and non-technical, so I think this would be an great bedtime read for anyone interested in the topic that isn’t in a rush. For faster and a more technical approach, one might consider John Pierce’s book.

Frequency bands are often used in analysis or input representation, for example, in mel spectogram, there are a number of bands of differing frequency widths used to represent the signal. In synthesis, frequency bands are also used. However, synthesizing a frequency band of non zero width is usually a noisy process. The most common way to synthesize this is by generating wide band noise, either with a white noise generator or an impulse, and filtering it to have fewer frequencies. For example, with a band pass filter or high-Q IIR filter. When narrow enough these will resemble sinusoids with some attractive roughness. FM can be used to create narrow bands as well, and be made more complex with a daisy-chained FM. In the music page, the piece “December 9” uses daisy chained FM and granular synthesis exclusively to create tones and rain-like sounds. Each of these has some drawbacks and advantages depending on your use case – IIR might explode, daisy-chained FM is unstable in other ways. They are all pretty neat – somehow it’s enjoyable to turn noise into something that resembles a sinusoid.

I want to describe another technique to create banded noise which I’ve used before that does the inverse, turning a sinusoid into noise. I’m fairly sure others have used this as well, but it doesn’t seem to be well documented. The basic idea is to start with a sinusoidal unit generator and linearly interpolate the instantaneous frequency from the previous target frequency to a random target frequency, with the target updating at an average but random duration that is inversely proportional to the width. The phase advances monotonically each step. The result is a band of noise that is a perfect sinusoidal tone at zero width, and white noise at full width.

This technique was described in my first academic paper at ICMC 2007 as follows:

It might be interesting to create some DDSP modules that implement this for DNN context.

I wanted to write in-depth book reviews for books that I enjoyed in 2020 at the very start of the year. Well, it’s already a month and I haven’t gotten to it.
So to kickstart it, I’m just going to make the task easier and just list some of the books I read in 2020 that were noteworthy with one or two sentences. And to narrow it down further, I’m just going to talk about stats-related books, because I read a few of them.

• Science Fictions, Stuart Ritchie (2020)

This is a relentless, even thrilling debunking of bad science, starting with the replication crisis in psychology and ending up touching much more than I expected. The stories are captivating, and the explanations focus on the system’s perverse incentives for fraud, hype and negligence more than blaming the individuals (although there is definitely shaming where it is called for). The criticism of Kahneman’s overconfidence in Thinking Fast and Slow was refreshing to read, because he (and Yudkowsky in his senquences) says something to the effect of ‘you have no other choice to believe after reading these studies’, which felt like it didn’t match the larger message about questioning your beliefs and updating them on new information. It is a good lesson, not without a healthy helping of irony that rationalists need to be told that they should be less confident that they don’t know everything yet. Another great criticism was about the hugely popular, and often unchallenged Matthew Walker’s Why We Sleep, which I just took at face value before reading this book.

• Statistical Rethinking, Richard McElreath, 2nd edition (2020)

This is an excellent introduction to Bayesian statistics, and pairs wonderfully with the author’s engaging, enlightening, and entertaining recorded 2019 lectures on YouTube as well as the homework problems on GitHub. Miraculously, McElreath manages to pull off a new video phenomena that mixes statistics with hints of standup comedy. There are no mesmerizing 3blue1brown-like plots, but McElreath picks interesting problems and datasets to play with, and breaks down the model into the core components that a newcomer would need to understand it. I also appreciate that the course is designed for a wide range of people, so there are very little assumptions about math backgrounds, but if you know calculus, information theory, and linear algebra, there are nice little asides that go deeper. It’s also great how up-to-date the book is – I didn’t expect to be so interested by the developments in Hamiltonian Monte Carlo that have come in the past few years, but it seems the field is undergoing rapid development. This book also helped me with understanding casual inference and confounders and is a great follow-up to the casual-reader oriented Pearl book mentioned below.

• The Book of Why, Judea Pearl (2018)

This book was the first thing I read on causal inference and causality in general. It’s a light non-fiction book that assumes no math background, and does a good job of explaining how to answer the question of disentangling correlation and causation. The book has interesting problems and examples, such as how controlling on certain types of data (confounders) can actually cause you to find spurious causation, and how to go about showing that smoking really does cause cancer when it is unethical to do a randomized controlled trial and there are tons of confounders. There is a fair amount of interesting history in it, and because of that, there are traces of personal politics that seem slightly out of place, but it doesn’t detract from the book too much. This book is sort of a teaser for Pearl’s deeper textbook, Causality, which I haven’t read. The Book of Why doesn’t really go into how you would create a model of your own, or how such a bayesian model would compare to the deep neural network/stochastic gradient descent models that are driving the computing industry today. Perhaps Causality covers some of these things, but I felt like it would benefit from a few comparisons between the popular frameworks with a toy problem that deep learning can’t solve. Still, it could be argued that this was not the point of the book. In any case it was an enlightening introduction that gave me other questions to pursue, which is the type of book I am after.