Less than half of the people identified as suffering from a mental health condition in the US received professional care in 2020, amongst whom shows a worrisome demographic disparity.

A rise in spending and popularity of mental health apps shows that people are looking for alternatives and more private coping strategies.

Mindfulness practices is one of the leading health strategies used by adults in the US in 2021, including meditation, journaling, deep breathing. (Statista)

A growing low-investment mindfulness practice is personal journaling. Chart: Searches for "journaling" grew over the last 10 years. (Google Trend, keyword: "journaling")

→ Journaling As A Wellness Practice

The most obvious reason for personal journaling includes to reflect and understand one’s own state of mind. However, less known was the compromised effectiveness of journaling if carried out incorrectly, potentially leading to excessive rumination and obsession with certain problems.

Effectiveness of journaling for wellness depends largely on the approach to writing: troubleshooting or ruminating, honest or avoidant, expressive or unemotional.

A study also shows that “objective writing” will not help reduce symptoms of mental illnesses, while “expressive writing” will. (Niles, A. N. et.al (2013). Effect of Expressive Writing on Psychological and Physical Health: The Moderating Role of Emotional Expressivity. Anxiety Stress Coping.)

Current diary and journaling apps do not offer much more than regular text editors.

Its cousin, mood tracking apps, oversimplify the process through self-reporting, choosing from a limited scale of emojis. Selection of “mood” is often prompted before writing, which goes against the natural process of understanding emotions through writing and only reaching a conclusion afterwards.

→ Journaling To Keep Memories

People also write to log memories and “immortalize” mortal happenings.

On the other hand, the writing is strictly of value to only the writer, unless they are comfortable with sharing or publicizing. Even to the writer, journals are not valuable unless time has passed. In that period of aging time, physical diaries are sitting and taking up space.

Diarists and journal keepers often end up wondering how to deal with old journals.

For journaling's critical use case, when the content should be kept extra private, writers often feel insecure and ashamed of keeping a “proof” of their thoughts.

Given the broad application potential for mood-based text to image conversion, narrowing down the features and bringing together one coherent product remain a big challenge.

Through a lot of brainstorming, weighing priorities using quadrants and trying to answer my own questions, moodi went from a web-app with only core functions to a mood diary mobile application with its own set of features.

This version of the Android app is comparable to a functional wireframe. I focused a lot more on the holistic idea of the app, so the UI is kept stripped down. This is largely due to time constraints and my newness to Flutter and Material App as a front-end.

XD wireframe

Android app

Database created for moodi is admittedly not the cleanest, but it did the job for the first version. This version has yet to include data logging for the “World” and “Summary” sections.

The app is built on the basic backend of moodi web-app. Since P5.js was the main requirement of the previous project, this version's art generation portion was done through Flutter’s Webview, making it a hybrid app.

To improve the app’s functionality, reliance on API should be minimized, making most functions local for convenience of use. Substituting out p5.js and html5 with Flutter's own version of canvas can reduce delays and failures due to too many simultaneous API calls.

Regular text-to-image models mainly focus on creating a literal interpretation of the text, generating images by focusing on visual descriptions or nouns.

Existing sentiment analysis models only generate keyword results, e.g., anger, sadness, joy…

The emotional perception of a visual piece is still too subjective and human for machine categorization, hence a lack of data for machines to be trained for sentiment-based text-to-image generation. It is my job to establish a bridge between what is available in machine learning, sentiment analysis of text, and a purely human act, art making.

In order to approach this vast and already vague subject of sentiment analysis, I looked to scientific studies of emotions. Two main models of human emotions are used to this day: categorical (every emotion is a different combination of 6 basic emotions) and dimensional (emotions can be described as different levels of valence, arousal and dominance). For the most attainable accuracy, I determined to include both approaches in the moodi algorithm.

Four ways of analyzing text were chosen including two machine learning models and two lexicons, set to run simultaneously. The moodi algorithm analyzes text by both context and affect, through different levels of text (aspect level, sentence level, document level).

Diagram: Analyzing deep learning model trained on Go Emotion dataset and its results by mapping each onto Ekman’s emotion categorization model

By not using a machine learning model that directly builds up images, I have the artistic freedom to determine what certain emotions will look like. Another advantage of this is I have total control of how each text-analysis model would affect the final outcome. On top of that, the generated image will not suffer from incompletion, absurdities and other typical output failures.

Processing Result →

confusion

My interpretation →

didactic, rapid, bouncing, random, entangled

Output Sketch →

Generated Feature

A different piece of generative art is assigned to each text-analysis result. Due to time constraints of having to populate at least 33 piece of generative art, many of the outputs are referencing other digital artists’ code with adjustments.

Generated Features (based on identified topic)

Generated Features (based on GoEmotions results)

Full mapping of algorithm