You could also consider "pseudo code" skills, or the ability to write almost in plain English how an algorithm or a query would work. Such a test would show the quality of the thinking and the approach to a problem, as well as how such a problem would begin to be solved by your applicant, regardless of if he or she actually possesses the skills in any given language to pull it off.
Be Prepared to Show Data Scientists the Money
As demand for data scientists increases, and as long as the supply of qualified candidates is being outstripped by it, salaries are rising. In almost any metro market in the United States, data scientists are receiving six-figure base salaries — obviously higher in high cost markets such as the West Coast. In Silicon Valley, in particular, multiple offers for a qualified candidate are not uncommon.
Don't attempt to pay below market rates for this position. Even startups are paying data scientists comfortable wages and giving them the ability to work on challenging new products, unlike their traditional modus operandi of loading up in equity positions and paying measly wages. Put simply: Don't cheap out and expect great talent.
Sign up for CIO Asia eNewsletters.