Architecting The PPAI 100 Scoring System
PPAI 100 is one of the most ambitious and potentially impactful projects in the Association’s history. Its creation was a collaboration of industry experts and external research consultants along with PPAI's own research manager, Robert Fitzpatrick, who helped create the final scoring system and shepherded the data.
The philosophy behind the program and its unique eight-category approach have many parents within the Association. Supplier and distributor volunteers helped to craft the survey questions that led to scoring in the innovation and responsibility categories, as well as the benchmarking information that has just begun to flow as a result of the program.
- PPAI Media spent more than a year studying what defines a top company in the modern promotional products industry, soliciting feedback from executives.
- Last December, the concept for PPAI 100 was approved and the process of modeling scoring scenarios began with advice from outside research consulting and the creation of numerous scoring models by Fitzpatrick.
- The methodology was finalized in the spring. After much testing with the use of past data and hypothetical scenarios, the final proprietary scoring matrix ultimately won approval for its ability to consistently produce the results most credible with industry veterans.
- Surveys were crafted by Fitzpatrick, based on suggested questions from industry experts inside the industry and within PPAI, and sent to suppliers and distributors known to have the mathematical potential to earn a spot.
To shed more light on the process that led to this sprawling new franchise, as well as what comes next, Fitzpatrick answered a few questions from PPAI Media.
PPAI Media: First of all, kudos and thank you for all the hard work that went into PPAI 100. It kind of became your life over the last several months. What was it like to work on?
Fitzpatrick: It was interesting to work on. Our objective was to create a list that was unique and credible, and also actionable for the industry. We found it difficult to decide which metrics accurately measure the overall strength of a company. Once we decided on the metrics, the project became easier, but I still didn’t know how it would pan out.
It had to be something our members cared about. In any kind of research project like this, which requires people to take surveys, your first fear is that no one will care and participation will be low. We ended up having great participation from the vast majority of the largest suppliers and distributors.
This project is something that was already a long way to being conceptualized before I was hired, so I knew there were high expectations. But I never doubted that we could rise to the occasion.
PPAI Media: What was the most difficult part of this?
Fitzpatrick: The most difficult part of the process was trying to figure out which metrics accurately measure the strength of a company and how much each of those metrics should be weighed. A lot of people weighed in from membership, the PPAI board of directors and PPAI staff.
You want to have the perfect metric to sum up a company’s online presence or the happiness of its employees. But at some point, you have to move forward with the best information and method that is available to you at the time.
PPAI Media: What do you hope the industry takes away from all of this?
Fitzpatrick: So, revenue is what the industry has been used to seeing as the only measure of a company’s strength. We wanted to dive deeper and look at it from additional angles that are aligned with PPAI’s strategic objectives, which come from our board of directors. And, of course, the board comes out of the industry itself. So this is really what the industry was telling us they wanted. These are the things that are going to make promotional products a stronger market in the future – universally valued and essential to every brand, as our vision statement says.
In a way this might be a window into which companies are best positioned for that future and to help create that future. Revenue is important and always will be. But things like corporate social responsibility and digital transformation are what will change the industry into something better, so we should highlight and celebrate the companies that make the industry better.
PPAI Media: What are you hearing so far? With anything new and different, and especially something that has this level of aspiration, there is bound to be some criticism along with the positives.
Fitzpatrick: Right, sure. We’ve expected that. I have not personally heard any critical questioning so far, but I do expect it to come as the industry has time to digest the lists. And I look forward to the critical questioning because that means we made the industry care. I expect people to challenge why certain metrics were used to score over other metrics. But, as I was saying before, at a certain point you have to go with the best information that is available to you.Anyway, we’re not preparing to debate everybody. We can defend why we did this or that, but what we’re mostly going to do now is listen. We’ll take what people say, think about it openly, and try to use it to make PPAI 100 better next year or as soon as possible. Just because we did it a certain way this year doesn’t mean it will always be done the exact same way. We’ll try and perfect the methodology a little more every year and keep things up to date with what matters to our members and what information they can share.
PPAI Media: And where is your head at now that your work is finished, the supplier list has come out and the distributors will be revealed next week, on June 27?
Fitzpatrick: First, I learned that I have more patience than I previously thought I did. I learned how to listen to other people’s ideas and pivot when necessary. In terms of learning about the industry through the experts we talked to, our research consultant and the team, I gained a new respect for everyone. They were always looking for ways to make it better. When we thought we had something right, they would question it and make it better.
Once my part of the project was finished, I was relieved, happy and ready to see what type of impact it would have on the industry. It does feel like a great pressure has been lifted. Now is the fun part.