This column is part of a continuing series by Alec Walker, cofounder and CEO of DelfinSia.
Outside of my work with software companies, I teach professional scientists and engineers how to write their own software scripts. People come to me wanting to automate repetitive work flows or perform data science tasks. I often run programs for corporate teams, and I often step in and do consulting, too. This has been going on since 2014. Over the years, I’ve taught thousands of people and worked with dozens of Fortune 500 companies, smaller operators, and professional societies. I’ve noticed interesting cultural differences between industrial sectors in their approach to dealing with staff software training. Here, I’ll try and synthesize them into a major insight and expound on the implications.
First, a reality check: The facts are the facts, regardless of the cultural lens. As Alex McCaskill, a geotechnical engineer in a major architecture firm put it, “Python is the new Excel. Not knowing how to use it in the job market today is like not knowing how to use Excel 10 years ago. Maybe you can get a job, but you’ll be struggling to keep up with the new standard.” As was mentioned in the 2019 AIChE Annual Meeting kickoff speech, “From our research, graduating chemical engineers with software knowledge on their resumes earn an average of $20,000 more today than those without.” As one client in an oil major put it, “As soon as I automate a task, I have a tool. Others start to use that tool, and they start coming to me with questions about tweaking it or making other tools. It sets a new standard. What would happen to all that momentum if I were to leave the team? My future is now more secure than my EVP’s.” Personally, I’ve seen my software training client base grow from a fringe of curious technology first-adopters to the mainstream, a swell of about 500% year over year. Check any major organization tracking the proliferation of software scripting and data science, and you’ll see three major trends:
- Software scripting is a skill for everyone today, not just those with job titles like computer scientist or software developer. Those with engineering or science backgrounds tend to get up to speed in about 16 hours of devoted effort with the right help.
- Contemporary data science methods are revolutionizing how data is used to inform decisions across industries ranging from finance to plumbing, with oil and gas smack in the middle.
- Python has outstripped other programming languages in pretty much every area of development, including data science. This is because it is especially easy to learn and powerful.
There are some interesting cultural differences between industrial sectors when it comes to accepting this reality and the resulting actions taken. Years ago, my boss in an oil company oversaw four assets worth several billion dollars; my direct manager when I started under that boss oversaw me and four other people. A very smart and kind colleague trained me how to do well evaluation and asset development optimization. Many parts of this process were rote and repetitive. I automated them. I taught my replacement how to use my automated tools and how to automate her own tools. I was 33% more productive than my predecessor in my role by my firm’s standards, and my replacement was as productive as I was. Eventually, my job for each week was done part of the way through Monday. I was bored. I went to other teams and asked for work. My manager told me to stop, and I was in trouble. I went to my boss and asked for more work. She told me that automating processes was impressive, but that, in her experience, automating took about as long as just doing the work herself and the repeatability just wasn’t there to justify it to begin with. Clearly, the data I had about myself and others suggested otherwise. Confused, I went back to my desk and stayed bored until I left the company.
Years have gone by, and I’ve worked with many oil companies since then. This denial of reality is still pervasive, despite the extraordinary pressure on the industry to improve efficiency. Interestingly, this denial of reality is not so pervasive elsewhere. Food and beverage companies, hotel chains, private equity companies, airlines, and even other industrial firms such as engineering, procurement, and construction firms (EPCs) and manufacturers are all working hard to get their people up to speed on writing simple software scripts to get things done faster and to make better decisions.
Perhaps surprisingly, technical EPC firms show much more widespread adoption of new scripting tools than do oil and gas companies. A structural engineering lead at Fluor (perhaps analogous to a Schlumberger) with whom I worked explained that, beyond the teams I got to know, engineers generally have been clamoring to catch up. A recent transplant structural engineering head at Dow (perhaps analogous to a Chevron) with whom I worked explained that teams are less likely to know about how the rest of the world is revolutionizing the way they get work done but that those who have gotten up to speed on software development are excelling within the firm. Perhaps also surprisingly, this distinction widens when we move upstream. The folks I know from Baker Hughes and Schlumberger are constantly racing to learn the latest Python libraries and how to use them in their work. There’s the denial I highlighted above, but then there’s also innocent ignorance. Many I work with in national and multinational oil companies have never heard of Python and, until I can show them what’s going on out there, assume that software scripting is limited to developers building consumer-facing web apps such as AirBnB.
Part of this ignorance is explained by the culture of the oil industry. Who wants to try something new when we attach our identity to convention? In Houston, when a bar advertised and then showcased robotic bartenders one Friday night, revenue dropped by 50%. In San Francisco, when the same event happened, revenue tripled; and it was a Wednesday. I know, because I helped organize the events both times. Why did Google abandon the oil industry? When Amazon was looking for the next headquarters, what was the No. 1 reason they passed up Houston without even adding the city to a short list? They believed Houston was too technically illiterate. The stereotype is that the oil industry attracts people who want to do all their learning and experimenting only once, when they first join a job. The stereotype is that they lean on the fact that something has worked in the past as an excuse not to explore what could happen better today, even when what has worked in the past is not working today (witness negative oil prices). Are these stereotypes true? I can say only that I can relate to why others hold them to be true.
Another part of the reason for this ignorance, I suspect, is the industry’s long history of simply buying access to expensive software that attempts to do their work for them. There are two ways to look at this tendency, both conveying that it is a bad habit and a doomed strategy. First, if every use case was the exact same such that a scaled software solution did the job right every time, why would the oil and gas practitioner have a job at all? If it is automatable, then get out of the way. Or, perhaps different use cases require customization and nuance to get just right. In this case, no standardized software is going to do it for you. At some point, you will encounter a situation that is not handled by an option in a dropdown menu. Here is a general rule: The more powerful a tool, the more specific its range of use cases. Third-party software has its place, but the surface area of that place does not cover the full range of need for software on the job.
Putting on my MBA hat, I think about it this way: If there is a type of work that a firm is paying to do again and again in the same way every time, software should step in and do it for them. If the use case is particular to the firm or part of the firm, then it makes sense to use software consultants or internal development. If, however, this repetitive work is done the same way by many firms such that a well-defined scaled problem exists, none of them should build software for it, because a third-party software company will do so and will do better than any of the individual firms could do. This should make sense intuitively. Who will build better email applications, Microsoft or Hess? The reason why Microsoft hosts email for companies such as Hess is that handling email is core to Microsoft’s business. OK, but what about a script that checks your email archives looking for emails from a certain team, and, if finding them, extracts a certain type of data and formats it in the way you need it formatted for the work you are about to do? Will Microsoft do that for you? No way. Will Avanade or DelfinSia? Maybe, if you can convince your boss that it’s a big enough problem across a whole team.
Here’s the thing, though: If neither of those options work, how are you going to do it? Are you going to learn to script software yourself, or are you going to open every email one at a time to cut and paste and format data one entry at a time for the next week? I ask because the woman down the hall is automating something similar in her own email archives, and you might want to learn to do the same. I ask because the company across the parking lot is training their whole team on how to automate these kinds of tasks, and their industry is now expecting this of them. And it might not be a distant future in which your industry expects this of you, too.
