A Better Way to Select White-Collar Workers
A guest post by Alberto Zaragoza Comendador
A Bet On It reader recently sent me this thoughtful proposal for better hiring of white-collar workers. Enjoy!
Mr. Caplan,
This email is motivated by your recent post on testing.
TL, DR: I think the current system for selecting professionals in many fields is bad. Fields like: finance, insurance, auditing, accounting...
The alternative I propose is frequent Excel tests. With those, companies can know how good future and current employees really are.
For context, I've been working in actuarial roles for three years. The current system is better than choosing employees at random but companies can do better. I've pitched my idea to a few people in insurance but it hasn't gotten traction - yet.
The current system
Working in the fields I mentioned involves both:
-Programming: R, Python... however, these tasks are not usually done by all members of a team.
-Wrangling with a million .csv, .xlsx and .txt files. This is usually done in Excel and the expectation is that all team members are good at it.
Companies don't have a standard test for these skills. Coding tests are used by "tech" companies but as far as I know they're non-standard. As for Excel, the only test I'm aware of is the world championship. Again, some tests definitely exist - maybe those LinkedIn multiple choice exams - but they're not relevant. Companies don't use them.
So instead companies in finance & insurance end up relying on:
-The candidate's experience and skills as stated in his CV
-Questions at interviews. These can measure sector-specific knowledge but not skill at using a computer
-If the profession is small (like actuaries) maybe also personal contact. It might be that the person interviewing you already knows you from a previous gig, or knows someone who knows you.
In the US there is also the factor of what college you attended. This is less important in Spain for instance - we have fewer universities and I think none of them is super-selective.
The closest thing to a perfect labor market: athletics
In athletics, especially running, there is no secrecy. Everybody who is in the business of hiring or sponsoring an athlete knows how good a prospective employee or sponsoree is. Your times are available on Strava, your running mates know what pace you can keep up with, your coach knows what you can do.
In running there are no miracles either. Nobody goes from being unknown to doing a 10k in 30 minutes. Before doing the 30 minutes that runner has been to a lot of events - his progression is public knowledge. There is the issue of doping but even that is unlikely to be a serious factor at elite levels: athletes are tracked very thoroughly, and one infraction can end a career.
Also, races are very similar to each other. If you do 1h 10m in one half-marathon, you're almost guaranteed to do a similar time in every half marathon. Main exceptions being:
-If you're injured: but then your athletic performance really is degraded and not for one day. Your bad time in the last half-marathon isn't a fluke - if you were to do another half-marathon a week after, your performance will also be bad. Clubs and brands will demand that you recover and do 1h 10m again before forking out cash for you.
-If you're sick (had a bad day): in that case you can do another half-marathon soon after and prove your worth.
Runners are being tested constantly. Contrast this with once-in-a-lifetime tests, like SAT, or exams for government jobs, etc. Yes, once-in-a-lifetime exams measure grit, perseverance, intelligence. But:
1: They measure how much of those qualities you had when you took the test (years or decades ago)
2: They also depend on memorizing factoids and regurgitating them on exam day. This is not really a skill useful outside of the exam context. For government job exams in particular, they largely boil down to which candidates had a partner or relatives who could support them economically for years - people who study for these things often do it as a full-time job.
3: They took only one snapshot of your performance. This is less reliable than a sequence of tests: maybe that one day you took the test you were feeling ill, the questions were particularly easy that year, etc.
These are the reasons the test prep industry is test-specific: you prep for the SAT, or the MCAT, etc. In fact you prep for the particular version of the exam that you think is most likely to appear that year. Whereas in running and athletics there is an industry of coaches and clubs that helps people get better performance generally, not focused on a specific event. The fact that the Berlin half-marathon this year will take a different roundabout than last year will not affect your training at all.
How white-collar fields can be more like athletics: Excel exams
Generating fake-but-consistent databases is easier than ever. Companies can demand that both prospective and current employees do a test a few times a year, setting aside a morning or afternoon for the purpose.
As you said in your post, the test should be designed so that getting a maximum score is almost impossible. I'm thinking of: 4 hours as a time limit, but you only get the top score if you finish in less than 2 hours. In the real world it's not just getting the numbers right that matters, it's getting the numbers right and on time.
I mention Excel because it's the most universal tool in business. Of course there could be variants of the exam that focus on insurance or whatever. There could be exams focused on other tools like R. But I think that the beginning has to be simple. Everybody in white-collar fields uses Excel, everybody benefits from knowing how to use it well.
Having standard exams is not a necessity, but I believe it would help everybody. If you scored well in an Excel test but it's one done only by Deloitte employees, how do you know it will be good at another company?
What an Excel exam would look like
These are some examples of exercises I believe should be in such a test.
Discounting
Everybody knows how to calculate a present value on the basis of discounted future cash flows. Not everybody knows how to calculate the present value projected for future years. There are two ways to do it:
-The clunky way: spot rates but adjusting the power, so that instead of discounting the 20-years-from-now flows by 1.0252^20 you discount them by 1.0252^19. This is asking for trouble, you're going to take more time and you're more likely to make a manual error.
-With a forward rate: a simple formula will output the present value for both the reporting date and any future date.
Why would you want to calculate the present value projected into the future? For example, you might want to compare PVs projected for:
1) 31.12.2024 with then-current-data
2) 31.12.2025 with the data available as of 31.12.2024
3) 31.12.2025 with then-current data
1) and 2) are just what I was mentioning
Also: real-world situation is you have a .csv with "discount rates" but it doesn't say whether they're spot or forward, and you don't want to ask (because you might look bad before the client who sent you the file, for instance). You should be able to tell from the numbers themselves whether they're spot or forward, without having to rely on what somebody said at some meeting.
In an insurance-specific exam, or IFRS17-specific exam, there is also the issue of discounting with the locked-in rate (the rate in force when the policy started) and the current rate. Discount curves normally have a 1-year granularity, e.g. if you're calculating a PV as of 30.06.2024 then you will apply the same current discount rate to all flows occurring between July 2024 and June 2025. But the locked-in curve is always January-to-December because that's how insurance cohorts work. So on one hand you will be discounting with curves that switch from one value to the next in January, and on the other hand you will be changing curve values in July. I bet a lot of exam takers will program Excel so that it changes the value for both curves in the same month.
Aggregating
A lot of what we do is summing values from different sources and granularities, then comparing the results. Some ideas:
-A dataset with about 30 fields, of which let's say 20 non-consecutive fields are the ones you're interested in aggregating. Here we'll see which candidates rely on VLOOKUP (which requires you to specify a column number, again this is asking for trouble) and which know how to use INDEX(MATCH).
-Some data will be in a .csv that looks ok when opened in Excel and pasted into your .xlsx, but then the formulas don't work. It should be second nature to every consultant: when you see a .csv, you open it in Notepad. And then you notice: the numbers have quotation marks, or there is a "." as thousand separator because the file originates in a country with that system, etc.
-If you want to do a SUMIFS but then in one of the conditions you want to include two or more values, how do you do it? Candidates who literally do one SUMIFS for each value are not going to get a top score!
Data cleaning
-Knowing how to use LEFT, RIGHT, and MID to extract characters. This sounds trivial but MID isn't. It may be that you only have the policy number as data, not the product category, but the category is implicit in the policy number because it's the part of the number between the "_" signs. Tough luck, product category doesn't have a fixed number of characters - how do you do this?
-The exam might ask candidates to flag rows with impossible values, e.g. an amount of Liability for Incurred Claims where the occurrence date is later than the reporting date (LIC by definition must have happened prior to the reporting date). Here it's critical to be agile with IF/AND/OR and to know the usual kinds of errors with dates. For instance your date might have a " ' " at the start transforming it into text: this means you can use LEFT/RIGHT/MID on it, but you cannot use >=<.
-Using pivot tables. Let's say your company has a granularity in which a product category can have several products, but a product can only be in a single product category. Pivot tables can show very quickly whether this is happening or a product is wrongly distributed across more than one category.
If it's such a good idea, how come companies aren't doing it already?
(Not an objection you will make, but one others will make)
It's the same story as luggage with wheels. Or why triathlon was invented in the 1970s - surely the means for the sport had existed for many decades. In fact it's less strange than the previous examples: desktop computers and Excel have only been in widespread use for three decades. The fact that something hasn't been done yet doesn't mean it's a bad a idea.
Please feel free to quote from this email, and feel even more free to pitch the idea every time you can.
Best regards
Alberto
The "standardized test" is an artifact of academia. It isn't worth that much to academics to evaluate prospective students, so they do it in a cheap, scalable, low-quality way.
Companies that employ white-collar workers typically invest much *more* per interview than a standardized test. They generally do not want standardized tests because they perceive their own hiring processes as a more expensive but superior way of evaluating prospective employees.
It's very possible that companies in some field have a generally bad system of interviewing, but it's hard to cut through all the noise because 99% of people who complain about the interview process on the internet are people who are bitter that they didn't get some job.
This is called "overrating your own skill/topic of expertise." Excel isn't even the most universally used app within the Microsoft Office catalog, let alone all of business, and most of the white collar professionals who use excel only need to use its most basic features and never use it for math above a high school algebra level. An excel test might make sense for an actuary or finance professional, but everyone else should be outsourcing financial/statistical analysis to those professionals. Possibly, being especially good at excel is negatively correlated with HR, management, marketing, sales, etc potential. It might even be a poor signal for finance potential beyond a minimum threshold, since finance success is about understanding the business and creatively enabling other departments, not speedy spreadsheet creation.