For my final project, I’m interested in creating something that can visualize not only the text itself, but also can show the emotions behind it, and somehow can empower my users.

Before I talk specifically about my project idea, I’ll talk a bit about my references: Dear Abby.



Dear Abby is America’s longest-running advice column, first penned by Pauline Phillips under the pseudonym Abigail van Buren, and today by her daughter, Jeanne—has offered counsel to thousands of worried and conflicted readers. Syndicated in more than 1,200 newspapers at the height of its popularity, it offers an unprecedented look at the landscape of worries that dominate US life.


I found this Dear Abby Archives and spent 4 hours on reading all those usual, unusual, sweet, bitter, weird, funny troubles.



4 hours later I was fascinated by the lady and her column, (I still am). So I decided to scrape all the 30years dearAbby data and to visualize them base on timeline, to show the changes of people’s trouble, to see if we are happier now, or things bother us are the same thing that bothered us 30 years ago.

Screen Shot 2019-01-02 at 11.22.45 PM



Over 78000 lines data are not good or clean enough for this project, I had to wrote a script to organize the dirty raw data. What I did on this stage are: I used regular expression to separated the daily questions from their answers, sorted them base on timeline and cataloged them to different topics.


User testing:

After the user test, I got a lot of valuable feedbacks from my brilliant classmates. They made me realized that compare to the changes in past 30years, the troubles themselves, the emotions, the humanities, the vulnerabilities are more beautiful and they deserve more attentions.

So I changed my project a little bit, I hope this project can comfort my users and empower them.

When someone feels sad about something, he or she can search the anxiety on my project, to see there were so many people had the same kind problems in past 30years, imagine they suffered from the troubles and they came over eventually. Hopefully, to see the troubles are not unique may help or comfort my users and can give them confidence to hanging there. 


Index building:

For the searching part, I used a JavaScript library called elasticlunr.js. I used scores to find out the most relevant results of the key words given by users. For example, a user is searching “husband cheating”, the script will go though all the data I have, if the both the key words “husband” and “cheating” appear in the main text, the score will be 2; if both of them appear in the title, then the score will be doubled to 4; if only one appears, the score will be lower.


Visualization D3.js :

I used tiny dots to show how many people in each year asked the similar questions. The different shades of grey shows how relevant to the searching key words: the darker the more closely connected.

users can move the mouse and hover on each dot to see the details of the troubles. Click each dots will transfer the user to the page of Dearabby’s reply.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s