Some things to learn from the British East India Company's growth and demise

On efficiency, hiring, incentivisation and how we confuse communication latency with analysis speed

The heroism that can win empire has no natural affinity with the wisdom and virtue that improve and consolidate it

Robert Grant

How to get large numbers of people to work together has been a Strange Loop obsession since the first few essays. What’s most interesting to me has been this isn’t a new problem, one that reared up its head in the late 20th century, and our supposed predilection for the complacent life.

Instead, since I wrote the canon, I’ve been wondering what the world looked like historically. After all, we’ve had large organisations before - the Roman empire, East India Company, Genghis Khan and Alexander’s conquering spree, the Chinese and Mughal empire. But in all the books and papers and articles about these that I read there seem some interesting and overlapping themes. This is a first look, and definitely not meant to be exhaustive.

So, to start with, one of the exemplars of administration that found success was the British East India Company. They ruled over an empire, created an empire in point of fact, and seemed to only be dislodged through the Queen’s velvet glove smacking them on the head, dissolving it in 1874 by government decree.

But almost two centuries of success is nothing to scoff at, even considering its ignominious means and the final end.

Chartered on the last day of 1600, the Company was a joint-stock corporation that grew to control most of the Indian subcontinent and wield a mercenary military larger than Britain’s army. Though a private monopoly, it was also a “quasi-independent state body,” writes Ratcliff, closely tied to the British Crown. The 1811 invasion of Dutch Java, for instance, was a combined Royal Navy and East India Company affair. And the British government would bail the joint-stock corporation out financially more than once on the “it’s-too-big-to-fail” model of state-capitalism.

It’s worth noting that in keeping with today’s worry about capitalist excesses, the reign of the Company over the ruled was so bad that the British Empire had to step in as as the good guys.

In 1857 the Indians rose in revolt against high-handed and oppressive Company rule – particularly its insensitivity towards their religions – and it took excessively brutal action by the Company's army to regain control of its possessions.

Following this failure of governance, the British state formally took over the East India Company's rule in India.

But in any case, let’s have a look at the organisation itself. At the top was the Board of Directors, 24 members who were elected by shareholders. below them were several different levels of management, including councils, committees, and officials. Each level had different responsibilities and authority levels. For example, the Board of Control oversaw all aspects of company policy, while lower-level officials were responsible for specific tasks such as trade or military operations. Their hierarchical system ensured decision-making could be centralized at the top levels while still allowing flexibility at lower levels, allowing quick response to opportunities or threats as they arose.

For instance the Company, once it started owning land, getting state appointments and engaging with the Nawabs in India, decided they needed court representation. But they of course couldn’t be full fledged ambassadors. And so they decided on a role called a Resident.

A Resident was effectively the cultural intermediary between the Company and the various Indian Raj’s they were with. They were knowledge conduits in both direction, had diplomatic duties both implicit and explicit, helped build bridges between the two people.

This level of responsibility is of course easily described in large, handwavy sentences, and impossible to describe in any meaningful detail. But the impact is extremely clear, whether that’s in identification and building of new ports, identifying new trade cargo and routes, or currying favour by lending their army to expand their empire.

So, they were undeniably effective, and grew fast, and survived for a long time. The question is how did they do this? Grow so fast and so effectively at a time when getting a message back and forth took months!

What did they do differently?

Efficiency

The first interesting thing is that while the structure, the charter and all that sounds about normal for any corporation, they had less than 200 people until the 1800s in the London office, part time clerks and all. That is efficiency!

In fact, there’s a wonderful dataset of all the clerks employed by the Company over a period of more than a century. (The category of clerk, which forms the basis for this dataset, includes a high proportion of the Company's home employees, ranging from low servants to senior managers with executive authority.)

Bear in mind by this time they had conquered plenty of land, especially Marquess Wellesley (1798-1805), and the Marquess of Hastings (1813-23).

In fact by 1856, with the annexation of Oudh, all the Indian subcontinent up to the Himalayas, and much of Burma, was ruled directly by the Company itself or by local allied rulers.

Even having a commanding army at their disposal, this is a small fraction of what it takes to do anything today.

Compensation and incentivisation

The Company seemed to pay its employees pretty well, had a ton of perks, and gave them the opportunity to profit alongside the organisation. For instance, at the headquarters as time went on, they did get more professionalised (or so it looks like), as the average incomes went up considerably.

Part of this is due to the fact that there were quite a bit of part time employment that started in the late 1700s/ early 1800s, when they were scaling quickly and needed help, and partly because the tenure of employees was increasing over time.

In 1815, a new clerk would start at £40 per year (compared to average worker salaries, about £28,590 today ($41,138). But that would rise: an employee working for 11 to 15 years would earn £220 a year (£157,200); after 39 years of service, £600 a year (£428,800).

The working conditions seemed to be quite variable. While in London they didn’t have any annual leaves and often worked 10-12 hour days (with a 2 hour lunch), the conditions in the factories were slightly different.

And at the Surat factory in the 17th Century, Ovington tells us, clerks rose at dawn, prayed, worked from 10 to 12, ate a large lunch, took a siesta, and worked again from 4pm to 6pm, followed by prayers, supper and relaxing by the water or a garden.

Moreover, in order to keep people engaged in their work, especially to take on perilous sea voyages and be away from home for years on end, there were plenty of perks.

While not quite Google cafeteria, the lunches and dinners served were exquisite! Free breakfasts in London, and on-site cooks (English, Portuguese and Indian) in factories, with plenty of alcohol to go around.

On Sundays and holidays, that menu could swell to 16 courses and include peacocks, hares, venison and “Persian fruits” like pistachios, apricots and cherries.

In one year at a factory in Sumatra, the 19 workers consumed “74.5 dozen bottles of wine, 50 dozen of French claret, 24.5 dozen of Burton Ale, 2 pipes and 42 gallons of Madeira, 274 bottles of Toddy and 164 gallons of Goa arrack”.

If that’s not enough, if you were a senior officer you got a pretty great stipend to spend on entertaining others.

In the early 19th Century, some official East India Company dinners cost more than £300, equivalent to some £19,850 ($28,554) today, while the Chairman received another £2,000 per year (about £132,300) for entertaining.

Also, they had an incredibly sexy headquarter to work in (marble bas-reliefs, pipe organ of a tiger devouring a European and jewel-encrusted gold throne of the Sultan of Mysore Tipu). Even the warehouses were elegant and stylish in the City.

Also, in one of the most interesting ways to ensure alignment, personal trading within the employees was allowed. Which means that someone who was truly enterprising could create their own bonus structure, and if successful make enough to comfortably retire thereafter.

Furthermore, the decentralization that took the form of private trade allowances, served as a motivation for the employees to stay for longer periods in the East, and help integrate the existing English settlements within a tight communication network while also caused a surge for exploring new ports.

It perfectly aligns the risks that the Company wanted their employees to take while not having to specify exactly how they took those risks. Talk about ESOPs.

The reason looking back at this era is so interesting is because communication latencies make almost any remote management impossible. Which means we get to see the benefits (and drawbacks) of having and needing high agency people to basically do anything with minimal oversight.

Hiring

EIC had to hire folks who could be relied upon to do their duty under reasonably autonomy and minimal instruction. At first, the Residents and other similar positions were fulfilled by military personnel. After all, the military was the most powerful part of the Company in a rather literal sense.

Soon after though, they started being brought in from civil services or those educated in specialised colleges.

Even taking into account the incredible amounts of nepotism, for everything including manual labour jobs, you needed a nomination from one of the company’s 24 directors. According to Huw Bowen;

Success was ultimately dependent upon connection and influence rather than the possession of any skills and aptitude for the post

You often needed to demonstrate commitment to the job by putting down cash, often £500 (this would be 100x bigger today). The amount increased by about 10x, or $5k, if you wanted to be the president of an overseas factory. What this did of course is that the jobs went mostly to those who could pay.

The Indian Service was confined to comparatively few families and he might reasonably expect to find relatives or friends of relatives at any of the three Presidencies, and these would give him hospitality and guidance. His initial employment would be as a copying clerk ; as such he would become familiar with the routine of the service before being given a post of responsibility.

Eventually they created the East India College to help educate the clerks, teaching all the essentials - “On the curriculum: history, the classics, law – and Hindustani, Sanskrit, Persian and Telugu.”

A step down from the leaders though the clerks were treated as if they were apprentices to be kept an eye on. For instance:

the depots were set up like monasteries or Oxford colleges, and workers slept, ate and prayed under the eye of their superiors.

The college had plenty of detractors, even during its life. It combined worries of high expense, the curriculum to train recruits, regarded as “a sink of immorality and vice, of disorder and irregularity”. As today, the major education method to create future Colonial officials had plenty of armchair theorists to naysay.

They even had their share of celebrity teachers.

Paying high salaries and granting generous pensions, the Court was able to employ a large number of prestigious educators, including in the first corps Thomas Malthus.

But the very movement from having appointments done by patronage to appointees educated specifically to be helpful in the local context is a major change.

---

Putting this together, the methods of hiring, incentivisation and operational efficiency seems to align reasonably well with some models of management today. However one major difference I can see is that with the communication latency being so high, it’s well nigh impossible to do anything beyond giving your employees autonomy.

The natural end result of the high communication latency was that the authority atop almost always had duties of punishment rather than the ability to charge exactly what a subordinate officer should do.

This, of course, is a highly charged strategy. After all, there are plenty of things for which forgiveness cannot be sought and punishment cannot fix.

Moreover, this is only possible if the people in whom you place the trust are capable of using it in the right fashion. There were the usual rules, for instance, on the need for strict separation of public duties from private enrichment, though by all accounts those were almost always ignored.

The really interesting bits were not just ornamentation though. They were integral to how the Company was set up and organised. They were definitely not afraid to move fast and break things, even when things meant entire regions and huge swathes of local population.

But despite this, from an organisational lens, the interesting bit is the dichotomy between the centralised planning and governance and rule-setting that they did from London, and the freewheeling nature of the local administration and the representatives. It’s also a dichotomy between the long hours and hard work that the yardmen put in in the London warehouses and Bombay factories, compared to the gunslinger approach that the Residents or Governors took towards finding new opportunity and capitalising on it.

We take it as an article of faith that the higher levels of information transparency and speed is a good thing. And it indisputably is. But a consequence of it happening has been that in many (most?) cases it’s been used as a reason to reduce the autonomy at the edge.

Today, in a similar situation to where EIC found itself, you would have a much larger corporate office, reviewing and validating each piece of information that came in from the field, analysing it, and sending out crisp commands that those Residents would have to obey.

Does this world function better? Well, depends on your assumptions on the inputs.

If the people being hired are pretty good, then no it’s not that much more of a benefit. If the people being hired can’t easily be trusted, then yeah, more oversight is gonna help a lot.

In the case of EIC at least, though they had strict governing rules they were rarely applied, which meant they were functionally quite decentralised.

Through the fostering of the utilization of social networks and the cohesive internal structures, the Company and its private army managed to integrate and expand the operations of the Company in the East, in a coordinated manner. It is noteworthy that the social networks utilized by the Company were instrumental in transferring crucial information among the employees, and thus resulting in the incorporation of new ports to the ever-growing trade network of the Company.

---

This same premise that worked for EIC might not exactly work in today’s more complex world. There is a world of difference between a PM working at Stripe trying to figure out the way to make a new product work and a new Resident using his own initiative and intuition to figure out the best way to expand the Company’s purpose. Where to deploy your private army is a reassuringly rare problem for the modern graduate.

The latter is much closer to freeform Partnerships or Enterprise Sales functions, where success is measured and measurable. Similarly, opening a new office in a different country, or starting a new skunkworks project, could be run with much more autonomy provided to those running it.

But even this process still runs with much faster and tighter feedback loops today! Iterations happen much faster, not just with the customer, but also internally. CEOs get to block new initiatives, GMs get much more frequent instructions from above, below and from her peers. Decision making is frequent, as are decisions made about decisions made by others.

If you think of any company as a network, with nodes linked through communication, then this is effectively a question of how much computation you believe should be done by the nodes at the edges, vs the central nodes.

The individual “nodes” don’t make many decisions on their own. The decisions they do get to make are often reconsidered, evaluated, analysed and often overridden under fairly short term time pressures.

How much of the decision making should be made central vs how much at the edges is a central question in organisational design. But no matter how we decide upfront, the end result inexorably pushes towards higher centralisation as the information latency decreases.

After all, if you are the boss and you get to see the information your subordinate sees roughly at the same time, the urge to make the decision over her head is quite high!

Some of this is even sensible. After all, if you can centralise analysis of multiple incoming data streams, you can create better action plans. But this is rarely universally true. Computational power with all new data-flows about everything happening at the company everywhere does asymptote, which makes central decision making ineffectual.

But amongst everything else, I think my biggest takeaway thus far is that we confuse communication latency with analysis speed.

Just because we can see more information faster, doesn’t mean we know how to make sense of any or all of it. The lessons are more complex than “treat your people as ends not means”, though that’s a big part of it. It combines:

• People knew when they could ignore the rules so they could do things that helped the Company

• They ran with a much smaller bureaucracy centrally than what we’d think necessary, even with multiple committees to devolve decision making into

• In many cases the employees could trade for themselves on the side, i.e., helping the Company helped themselves - like a franchise model in miniature, or even a version of equity shares

• When you ask a lot from employees, its good to give them a lot - whether its salary, perks, nap rooms or better working conditions

• You need at least a few people who are freelance gunslingers to go unlock opportunities you can’t know about, to explore more of the territory

• There’s an increase in focus on assessments (and punishment) after the fact rather than permission to perform an action

It’s an interesting test of how to design an organisation to stand the test of time and expand across multiple continents, and to ensure you get maximum use of employee creativity by giving them autonomy, while still creating enough safeguards to align them with your own interests.

All that said, one final note, from the famous Charles Lamb, the eminent essayist, who worked for the Company. As he wrote to William Wordsworth.

I grow ominously tired of official confinement. Thirty years have I served the Philistines, and my neck is not subdued to the yoke. You don’t know how wearisome it is to breathe the air of four pent walls, without relief, day after day, all the golden hours of the day between ten and four, without ease or interposition. O for a few years between the grave and the desk!

Some things never change.

Addendum: this tweet is precisely the point!

Fruit of Tears @fruitoftears

Who had more administrators, the British Raj or Yale circa 2022?