The team
My role
I was the product designer on the project, which meant I designed all the screens, leveraged user testing sessions and interviews to make informed design decisions, presented to stakeholders, and set the long-term vision of the feature.
Other team members
Also on the team were a staff-level designer who provided design feedback, a UX researcher who ran user interviews and conducted studies, a product manager, and several engineers.
Users and audience
Stack Overflow users are developers and technologists who want to find answers to their questions or compare technical approaches to solving a problem. Most of them visit the site while at work and are searching for immediate answers, however there's a smaller contingent that volunteer their time answering questions and curating the site's knowledge base.
Prior research
There were some insights from prior rounds of research that informed the direction of this project:
- Users want to feel a sense of community. Stack Overflow's users have a desire to connect more with one another and feel a stronger sense of community than the platform currently allowed for. It was determined that the most successful communities feel like "small towns" rather than "big cities."
- Stack Overflow's strict rules around Q&A deter many users from participating. There are strict rules for the existing way to engage on the platform: asking and answering objective questions to create a comprehensive knowledge base. This is overwhelming—especially to new users—and blocks users from getting information that would help with decision making around more complex technical problems.
- There's a strong desire for subjective content on the platform. Tons of questions are posted then closed due to being subjective despite having significant upvotes for both the question and the answers provided.
Challenge
How do we increase engagement on the platform and provide something generative AI can't?
Our hypothesis
Providing a "small-town" space for subjective discussions will increase a sense of community, break down barriers to participation, and fulfill an unmet user need, and therefore increase participation on Stack Overflow.
Design process
The exploratory sprint
In a week-long cross-functional design sprint, the team came up with many ideas that would increase both a sense of community as well as engagement. We then tested two of the most promising concepts in over 8 hours of in-depth user feedback sessions, with representation across a large spectrum of user types.
I designed one of the solutions tested, which had a working name of "Communities." The concept allowed users to have technically-focused but subjective discussions in small communities within the platform as a whole. I focused the design around showcasing the people writing the content rather than just the content itself, as well as providing new ways for users to engage.
Promising results in user testing
We tested this concept in over 8 hours of user interviews, and the results were promising: the consensus was that the feature would positively contribute to both their knowledge sharing experience on Stack Overflow as well as their sense of community.
Key learnings
- The community must be visibly active. Several users said if they visited a community that appeared dead, they'd never come back.
- The content must be high quality and will need a set of standards and guidelines to maintain that quality. Further research helped us define this set of guidelines.
Remaining questions for further research
- What should the tone of the new space be? Should it lean more friendly and casual or more professional?
- What should the main method of interaction be? Users were split between wanting to vote on content (just like they did on the rest of the platform) and wanting to "like" it or react with an emoji.
- How much should the experience stick to existing Stack Overflow norms and signals of high-quality content? Should these posts have titles, tags, and scores?
After getting leadership's buy-in on this concept, we began work on what became known as "Discussions." Because Stack Overflow already had sub-communities around specific topics called Collectives, it made sense for Discussions to slot into those.
Strategy and approach
Because participation on the platform was continuing to decrease, we wanted to get an MVP of Discussions released as quickly as possible to test our assumptions and involve the community early on in the process. However, we didn't want to lose sight of our grander vision for what Discussions could look like in the future, so we had two workstreams going in parallel.
Future vision
- What the product could look like in the future
- Would feature an updated design system to be used here and eventually rolled out to the rest of the platform
- Would act as a guide for the MVP and subsequent iterations
MVP
- Allow us to test our assumptions in as little time as possible
- Collaborate with the community sooner
- Use the existing Stack Overflow design system
- Base it on an existing product and make as few changes as possible
- Design, build, and launch in 5 weeks with a team of 3 engineers
Future vision
I chose to design the future-facing version first in order to inform which features we'd prioritize for the MVP. Leveraging what we learned from the first round of user testing, I emphasized community activity and content quality.
One major question still remaining was how much this space should resemble social media vs. follow existing norms on the platform. This would affect the visual tone of the space as well as the main method of interaction. To test this, I created two versions of what a possible future of this feature could look like, and we put them in front of users.
We also tested different versions of the post summary component, ranging from following existing norms exactly to being extremely like social media. Ultimately, users preferred something somewhere in the middle, but still leaning towards using existing site norms.
The final post summary component looked like this:
I applied these learnings to evolve the project further into its final version:
MVP
Ultimately, we wanted to test our assumptions by releasing something quickly and observing what happened. This led to us taking an existing feature that enabled private messaging, duplicating it, and changing only what was necessary to release an MVP. We would involve the community early, see how people use the feature, gather feedback, and then figure out which additional enhancements would be most beneficial.
We were pretty ruthless when making cuts. For example, sorting and filtering discussion posts were cut. The idea was to see how many discussions ended up being posted and then only spend the development effort if there were enough that sorting and filtering would be beneficial.
We also created onboarding and discovery functionality, including a banner that appears when a user visits Discussions for the first time as well as a banner promoting Discussions that appears on other pages:
Outcome
The participation rate was nearly 3x our goal, which was based on previous benchmarks.
35% higher weekly retention rate among contributors of Discussions vs the rest of the platform.
In August 2023, we released the MVP of Discussions to a single sub-community on Stack Overflow. After an encouraging two weeks, we expanded to several other sub-communities. With participation and retention outperforming expectations, we set our sights on adding functionality from our backlog that would make the experience better, as well as functionality requested by the community.
This includes:
- The ability to sort and filter discussion posts
- The ability to sort replies on discussion posts
- The ability to follow a discussion post or thread of replies to be notified of new activity
- A way for users to share posts and replies to social media
- A way to see whether a reply has been edited, and what the edit was
