How AI Helps (and Doesn’t Help) with My 70k Reader Newsletter

We're in a weird halfway world where AI is both somewhat competent, and unreliable.

Welcome to the Scarlet Ink newsletter. I'm Dave Anderson, an ex-Amazon Tech Director and GM. Each week I write a newsletter article on tech industry careers, and specific leadership advice.

Free members can read some amount of each article, while paid members can read the full article. For some, part of the article is plenty! But if you'd like to read more, I'd love you to consider becoming a paid member!

AI has been changing the tech landscape dramatically over the last few years. I’ve been repeatedly asked about how I use AI, or how I think AI could be used today in writing and social media.

That’s certainly difficult to answer in the limited text space we have available in a newsletter. But I’m going to take a swing at it. And since I just finished my article (scrolled back to the top to make this note), I’ll say that this one turned out to be quite the doozy. It took me a good amount of AI usage, testing, etc.

Uniped seagull in New Zealand. I’d feel bad for him, but he seemed healthy and happy. Photo credit: Me

Four disclaimers:

1. For my examples, I will use Claude and ChatGPT. I’ve used various AI models, and I currently think Claude and ChatGPT are the most likely to return useful results. So I’m using them for my examples because it’s easiest. I don’t plan to compare Claude and ChatGPT because that’s not the point of this article. Each example will just be from one of them randomly.

2. I’m quoting anecdotes, not data. I’ve spent many hours prompting various AIs. So many hours. At various times they’d give better or worse results. I’m looking to show a typical response, not the best or worst.

3. AI changes all the time. I try using AI essentially any time a major release happens because their capabilities are changing so quickly. So if you read this in 12 weeks, it’s possible that things have changed.

4. Success for me means a substantial savings in time. If it takes me 5 minutes to prompt AI to save me 7 minutes, it’s not worthwhile. And if I prompt it, and then have to re-write it regularly, again, it’s not a success. Or if it takes me 12 hours to create my own self-trained AI to save me 5 minutes, it really is unlikely to be worth my time. So my version of success is a slam dunk, “I save a ton of time”.

I have a request for any readers who feel they’re better at AI and AI prompting than I am. Can you look at my requirements, and get better results than I do? I’d love to hear from you in comments or over email.

What is my writing business?

Let’s start from the product requirements point of view.

I write a paid newsletter article once a week. My articles are purposefully evergreen, meaning that they’re almost never time sensitive. An article I wrote in 2022 should be relevant now. Which is a key data point for understanding my social media strategy.

Every Monday, around 6 something AM, my newsletter is sent to around 70k people. Everyone gets a portion (30-50%) of an article, and a few thousand of those people pay me monthly / annually to read the full article. (Thanks everyone!)

I get new readers through two main methods of discovery:

1. Substack has a recommendation feature. Other newsletter writers recommend my newsletter, and people end up subscribing (for free or paid). I don’t go out of my way to make this larger, although I’ve seen newsletter writers optimize this more than I do.

2. Social media posts, primarily LinkedIn (for me). I could write a very long post about what I’ve learned about social media. But the primary takeaway is that my newsletter slowly shrinks if I don’t write social media, and slowly grows if I write social media. So it’s my controllable growth lever.

This means that my work consists of two major bodies of work. An article and social media to advertise the overall newsletter.

My writing work, and where I could potentially apply AI.

Let’s break down those two bodies of work into the more precise elements.

1. Idea Generation

Every dang week, I need to come up with an article idea. Seriously, it’s no joke. I’ve had great coffee meetings with someone, where I jot down enough article ideas for a few weeks. At other times, I spend most of my writing time brainstorming. Once I have an idea I like, the writing itself tends to take less time than thinking.

Personally, the ideas come from a mix of just thinking about the past while staring at a wall, reading Reddit posts on the various CS related Subreddits, emails / comments from readers, and discussions with real life people.

2. Article outline

Once I have an idea, it immediately begins to break into an outline in my head. Funny enough because writing outlines in school felt like a waste of time. Turns out it’s not a waste of time at all. As a side note inside a side note, one reason that outlines are useful is to ensure that an article idea is a fully fleshed idea worth writing 3k+ words about. More than once I’ve had a good idea, and realized that it’s more of a 750 word idea. Which might work for social media, but not my newsletter.

The structure of the article might be a narrative story following a timeline (A Day in the Life of a Senior Manager at Amazon). Or it could be a listicle (12 Things I Learned in Big Tech). Or it could be some version of (without a cute name for it) an educational article (How to Both Embrace and Defeat Imposter Syndrome).

3. Article body

The most obvious part of the writing process is filling in that article outline with content. I sit there, and type. Thankfully, this is the part I like, and believe I do well. Including the typing. I’ve always been good at typing.

I also insert in a couple of personal photos, but that’s barely work. It’s mostly looking through our photo collection randomly, and attaching a couple of photos that strike my fancy.

4. Article quality control

I’m listing this just to explain that I don’t do much of this. For philosophical and practical reasons, I try not to do much post writing editing. I write, I usually skim the article, and then I publish. For my style of article, I want it to be natural, and not overly edited.

5. Social media content generation

I need one to six+ paragraphs written for each social media post. Each post needs to include text, and a link to a corresponding/relevant article. My general social media strategy is to make useful social media posts. When a reader finds them useful, they’ll like/share on social media, increasing my post visibility. They also might click through to the article, and pay to read the whole thing. Which is where the money comes from.

I wrote software which gives me a list of random archive articles (my content is purposefully evergreen as I mentioned) to write content for. Therefore, my main work is to turn a previously published article into a social media post.

I’d say that social media content generation is the hardest work that I’m not interested in doing. I’d rather just write articles, but it’s critical for me to write high-quality content on social media, or my newsletter slowly declines. Which is why this section of the AI section below is the most important (to me).

My overall thoughts on using AI and this article.

I enjoy writing, and I do this as a hobby. Having AI write my articles for me would defeat the purpose. So that’s not something I’m interested in at all.

However, I’m not against using AI to help with other aspects of the work involved.

For the purposes of this article, I rate the usage of AI from 1 to 10. As a (usually) data-driven person, I’d love to tell you the exact criteria by which I’ve rated AI usefulness. Except I can’t. Because in all cases it’s extremely subjective. Like, the most subjective thing you could possibly get. So you’ll just have to deal with me giving subjective number ratings.

It’s entirely likely that in my examples below, I’ll explain something I liked (or didn’t like), and a reader will disagree. That’s cool. I’m fine with having readers who have bad taste. I’m still happy to take your paid subscription if you have one.

AI doing Idea Generation?

Let’s start at the most basic requirement. Let’s see if AI can give me ideas out of relatively nothing.

Idea Generation from scratch

Prompt: “I write a newsletter. My newsletter is intended to educate both technology veterans and more junior folk on how to grow their careers, build their leadership capabilities, and level up their interviewing skills. I sometimes write narrative articles explaining what a specific type of day was like. I sometimes write list articles. And I sometimes write informative/educational articles. Could you please give me 5 unique article ideas I could write?”

Summary of ChatGPT’s response:

One - A Day of Saying No: What It Really Means to Be a Senior Leader

Two - 5 Interview Questions That Reveal If a Company Is Worth Your Time

Three - The Middle Manager's Guide to Leading Without Burning Out

Four - Career Growth Is Boring, And That’s the Point

Five - What I Wish Someone Told Me Before My First Senior Role

I’d give that a 6 out of 10 rating. It’s fine. I’ve written at least 3 out of 5 of them, which alone means that it’s decent at generating ideas. I followed up and asked for another 5, and to save room, I’ll just say that they were all fairly generic article ideas (not as good as these 5).

However, it has no context on what I’ve written, so it keeps stomping on my prior written articles. I’d also say that for personal taste, I’m not particularly excited about writing the ones I hadn’t written, so it’s lightly missing my taste mark.

But what if it starts with some knowledge of my newsletter? Maybe it could be more on-point?

Idea Generation from context

I asked, and ChatGPT seems to have context on the existence of my newsletter. It seemed to know the tone of my writing, and what I’ve written about. When prompted, ChatGPT confidently gave me 10 article titles I’ve used in the past. Weirdly, some were accurate, and others were absolutely made up (but could have feasibly been article titles based on tone and phrasing). Can’t make AI stop hallucinating, apparently.

So I asked it for 5 more ideas, now that it has context.

One - Stop Optimizing Metrics That Don’t Matter

Two - What Senior Engineers Actually Do All Day

Three - The Dangerous Seduction of Process

Four - You’re Not a Victim. You’re a Participant

Five - Your Title Doesn’t Make You a Leader. Your Behavior Does

My rating is 5 out of 10 (slightly worse than blind idea generation). My general feeling is that the ideas generated are highly related to the article titles it regurgitated when I asked it for my recently written articles. Unsurprisingly, giving AI more context actually makes it less creative.

I have also tried giving it a list of 50 past articles, and give me a few more which are not repetitive. It was still too derivative.

Idea generation summary

I think the biggest issue with AI idea generation is that it doesn’t know my experience. If it gets lucky, it’ll stumble on something which I haven’t written about, combined with something I’m interested in. But that’s a tough intersection to find.

To get better than blind idea generation, I might need to have a better prompt which explained the universe of possible article topics, and then somehow explain to AI what I’ve already written.

But anyway, my overall grade is a 6 out of 10.