The just-published Culture 500 Study by MIT Sloan Management Review and Glassdoor analyzes 500 of the largest firms in America and provides findings on corporate culture. The study anoints 21 companies as “culture champions”, recognizing them for cultural excellence.
Some of the most famous firms with trillion-dollar market capitalizations do not do well in the study. Google is identified as one of the least agile companies in America. Microsoft does not do well on Collaboration, or Apple on Customer Orientation. It might be no surprise that Verizon does not do well on Execution, but it is striking that Apple and Google “fail to make the top 70 on Innovation.” Given the troubled history of studies on corporate culture, a closer look at the study’s methodology is warranted.
The Culture 500 Methodology
The Culture 500 was created by Donald Sull, a senior lecturer at the MIT Sloan School of Management and co-founder of CultureX; and Charles Sull, a co-founder of CultureX.
While Glassdoor is a treasure trove of employee submissions about what is going on in large corporations, it has been difficult to access its riches. MIT Sloan Management Review and Glassdoor deserve congratulations for collaborating, pulling the data together, and providing a search tool to enable access to it. The tool is accessible here.
The Culture 500 interactive online tool applies artificial intelligence to analyze 1.4 million of Glassdoor’s reviews for 500 of the US’s most powerful companies and ranks corporations across what it calls the “Big 9”: Collaboration, Integrity, Agility, Diversity, Customer Orientation, Execution, Innovation, Performance, and Respect.
When I talked with Charles Sull this week, he explained that MIT Sloan Management Review has entered into a partnership with Glassdoor. CultureX has developed technology for analyzing accurately hundreds of different cultural topics at scale. It took around three years before the launch of the first Culture 500. It was an intensive process, which involved supervising and fine-tuning the accuracy of tens of thousands of different terms with cultural significance. This is now the second iteration of the Culture 500.
While people often think that Glassdoor reviews are all negatively biased from disgruntled employees, Sull told me that more than half of the reviews are four- and five-star reviews (51%) and only 10% are one-star reviews. People speak candidly on the platform, which has the advantage of being directly comparable because everyone is speaking on the same platform.
A Glassdoor review involves one-to-five scale ratings on matters such as the company overall, its culture, and its CEO. But what really interests the researchers is the free-text submissions. People are essentially asked: what is working well about this company? What is not working so well about this company? And then what is your advice to management? The free-text submissions provide the richest source of cultural information and that’s what the technology platform is designed to measure.
The insights of the Culture 500 come from analyzing the free-text comments. Hundreds of different types of cultural topics are measured. But for the purposes of the Culture 500, there are nine different values: Agility, Collaboration, Customer Orientation, Diversity And Inclusion, Integrity, Innovation, Performance, Execution, And Respect. And so whenever someone mentions one of those different cultural topics, which they do in a variety of ways, the study picks up on that and notes whether they are speaking about it positively or negatively. Then the study compares the ratio of positive to negative in the way people speak about these values as well as how frequently they speak about them.
The Promise And Disappointment Of Culture Studies
Culture studies first rose to prominence in the 1980s at a time when there was a suspicion that Japanese firms were doing well because of their strong corporate cultures. It was hoped that corporate culture might be a kind of Rosetta Stone that could unlock the secret of how an organization would perform. If one knew the values and norms of a corporation, one would have a hidden key to the future.
The utility of providing such a Rosetta Stone depended on being able to reliably identify and measure norms and values and show a relationship of correlation or causation with performance variables.
The track record of hundreds of studies of corporate culture has however been disappointing. It has resulted in a lack of unity and precision in deﬁning and measuring culture. This ambiguity has constrained progress in both developing a coherent theory of organizational culture and developing replicable and valid ﬁndings.
A Mix Of Culture And Performance
The Culture 500 study, Charles Sull told me, follows Professor Charles O’Reilly and Professor Jennifer Chatman in defining culture as a set of norms and values that are widely shared and strongly held throughout the organization. These values express what’s important to employees.
What is unusual about the Culture 500 study is that, of the nine variables chosen, only three are unambiguously norms and values: collaboration, integrity, and respect.
A fourth variable—diversity—could be seen as a mix of performance and a value that corporations aspire to. It is upheld by a series of norms e.g. “provide safe spaces for discussions of inclusion”.
The other five variables appear to be themselves performance variables: agility, customer orientation, execution, innovation, and performance. On these variables, the study is summarizing what employees have written about them. The researchers believe that these performance variables are also aspirational values with an associated set of norms.
Yet it is not obvious that the employees always have the best information or judgment on performance matters. In fact, customers are also able to tell us about customer orientation, innovation and execution, sometimes more accurately than employees. Financial information and accounts also shed light on performance and execution. (The researchers see these performance variables as also aspirational cultural values.)
In a large corporation, employees necessarily have a worm’s eye view of the world. They can see very clearly what is happening in their tiny piece of the corporation and they know when they are being well or badly treated. On those issues, they have privileged insight.
But they do not always have a wider perspective on how customers are being treated or what is happening on overall performance or execution. A random selection of employees’ written conversations on these issues might, or might not, have significance. It isn’t always obvious that applying artificial intelligence to summarize and rank the comments elevates them to anything more than employees’ comments, of which the relevance and validity remains to be established.
The researchers on the other hand believe that employees understand these performance issues better than anyone. They believe that multiple worms’ eye views may accumulate to a bird’s eye view. In some cases, employees are the ones treating the customers well or poorly. They see the study as examining how a company rewards results through compensation, informal recognition, and promotions, and whether it deals effectively with under-performing employees.
Company Culture Vs Subcultures
Academic writers, such as Schein (1992) and Deal and Kennedy (2000), advanced the idea that organizations often have very differing subcultures. Although a company may have its “own unique culture,” in larger organizations there are often co-existing or conflicting subcultures because each subculture is linked to a different management team.
Differences between different parts of the company may be even more striking. At Amazon, for instance, management practices in the high-end intellectual work of software development (around 15% of the firm) appear to be the polar opposite of what goes on in Amazon’s Fulfillment Centers where the work is mainly manual and tightly controlled. In cases like these, one has to wonder about the meaning of a composite term such as “the culture of Amazon”.
Future Evolution Of The Culture 500
If the Culture500 study is to be more useful in future, it might evolve along the following lines:
· It could distinguish more clearly between culture and performance variables.
· It could recognize more clearly the differences between subcultures in a corporation, at least where there are stark differences.
· It could recognize more explicitly both the strengths and inherent limitations of employee viewpoints on performance variables.
· It could recognize more clearly that a corporation is a complex adaptive ecosystem of managers, talent, customers, suppliers, shareholders, and society, in which employees’ comments are one source of insights but not the only key to understanding corporate culture or the organization. (This is a topic the researchers are already working on.)
· It could give greater recognition to the insight that a corporate ecosystem is more than a bundle of unconnected values and norms, and constitutes a highly integrated set of arrangements that can only be fully understood as a system.
And read also: