Conference Post Mortem Analysis

What 763 Survey Responses Actually Said About the Event

How ORRO used Natural Language Processing (NLP) and sentiment analysis to turn post-event feedback into clear, defensible priorities for a national conference organizer

The Challenge

Every conference organizer faces the same uncomfortable moment after the event: the survey results arrive, the rating averages look acceptable, and yet the open-text comments tell a messier, more complicated story.

Which complaints are isolated grumbles and which represent a real pattern? Which positives are genuine strengths to build on and which are just polite noise?

The organizer of a national economic development conference in New Zealand had exactly this problem. Their post-event survey collected hundreds of open-text responses across more than 27 questions — covering sessions, speakers, networking, the conference dinner, venue, and future themes. The quantitative ratings told them the conference was broadly acceptable. What they needed to know was why, and what to do about it.

Reading every response manually was not a realistic option. Neither was treating the data as anecdote. They needed a rigorous, scalable method to turn qualitative feedback into clear, defensible priorities.

What We Did

We applied a multi-stage NLP pipeline to 763 survey responses drawn from 32 organizations across New Zealand, including regional development agencies, local councils, and central government bodies.

Our approach combined three methods:

1. Sentiment Classification We used a BERT-based sentiment model to classify each response as positive, negative, or neutral — applying a confidence threshold to ensure only reliable classifications were included in the analysis.

2. Topic Classification We applied a multi-label topic classifier, trained on conference feedback data, to assign each response to one or more of four core dimensions: Content, Organization, Venue, and Networking. Responses could carry multiple labels, reflecting the reality that a single comment often addresses more than one aspect of the attendee experience.

3. Frequency-Sentiment Matrix Analysis We mapped each topic against two dimensions simultaneously: how often it was mentioned and how positively or negatively it was discussed. This matrix revealed something that summary ratings never could — the gaps between what people talked about and how they actually felt about it.

The result was a structured, visual analysis of where the conference was genuinely succeeding, where it was quietly underperforming, and where attendee expectations were going unmet.

What We Found

The Most Discussed Topic Was Not the Most Positively Received

Content dominated the conversation — mentioned by 95.9% of respondents. But it received the lowest sentiment score of any topic: 0.25 out of 1.0.

Organization told the opposite story. Mentioned by just 10.2% of respondents, it scored highest in sentiment at 0.90. Networking sat in the sweet spot: mentioned by 53.1% of respondents and scoring 0.52.

In short: the thing attendees cared most about was also the thing they were least satisfied with. And the operational excellence the organizing team had worked hardest to deliver was nearly invisible in the feedback — not because it had failed, but because when logistics run smoothly, people don't mention them.

The Overall Rating Masked a Significant Insight

The conference averaged 2.35 out of 3 — technically positive. But 47.8% of respondents sat at the midpoint, rating the event a 2. That is a different management challenge than a bimodal distribution of enthusiasts and critics, and it calls for a different response. The data revealed a large cohort that was neither won over nor alienated — exactly the group most worth understanding and most worth winning.

The Qualitative Responses Explained the Gap

Digging into the open-text comments behind the content sentiment scores revealed consistent patterns:

Several presentations were perceived as promotional rather than educational — attendees wanted authentic lessons learned, not polished success stories. Sessions were information-dense without enough breathing room between speakers. Some felt the technology focus was over-weighted relative to appetite for broader thematic variety. The second day's speakers and the awards night were consistently highlighted as standout moments.

These were not random complaints. They were a coherent signal about what the audience had come for and what the program had not fully delivered.

What This Meant for the Client

The analysis gave the organizer something rare: a ranked, evidence-based set of priorities rather than a flat list of equally-weighted suggestions.

The clear finding was that the ceiling on overall satisfaction was content quality — not venue, not dinner format, not networking structures, which were already working well. That is where attention and budget needed to go.

The data supported three concrete actions for the following year's event: strengthen the speaker brief to require presenters to share genuine lessons learned including failures rather than polished organizational stories; introduce structural variety into the program through shorter slots, more breakouts, and scheduled pauses; and broaden the thematic scope modestly while retaining the technology focus attendees valued.

Equally important was what the organizer could stop worrying about. Organization scored exceptionally well — confirmed by data rather than assumption. That matters when budgets are being allocated and every decision has an opportunity cost.

One attendee described the conference as the best event they had attended in four years of participation. The data helped the organizer understand precisely what drove that response — and what stood between the rest of the cohort and feeling the same way.

What This Means for Conference and Event Organizers

Post-event surveys generate more qualitative data than any team can manually process. Most of it gets skimmed for a few representative quotes and filed. The patterns — the ones that would actually change next year's program — go undetected.

ORRO's approach makes that data fully analyzable: every response classified, every topic tracked, every sentiment measured. The result is not a summary of what people said. It is a map of what they meant.

Interested in what your event survey data could tell you?

ORRO builds bespoke text analytics solutions for organizations with complex, unstructured data challenges. Every engagement is custom — we work with your data, your questions, and your domain.

Page updated

Report abuse