Implementing The Simulation

To re-cap part 1, having been interested in a forum post, I decided to test out writing a simulator for the  Monty Hall problem for use as an interview exercise.

We need to give the user a method for specifying the number of games, the number of doors to use in a game, and display the results. This seems like it fits into a classic, resource based Rails solution with a restricted number of views.

Lets start with the model.

Database Constraints Or Rails Validations

I decided to create a Simulation model which stores the information we need: game_count, door_count, first_guess_win_count, second_guess_win_count, log. Aside from log which we will discuss later, those fields should be self explanatory.

As we always require the door count and game count to be able to simulate the game, these should be required fields. So now we enter the great Database constraint debate, to code constraints on your table or within your application. Lets take a look at the tests and see what happens.

I create the SimulationTest class and a migration.

If you don’t add the validation to your migration and put them in your model, the tests will work, everyone is happy, its time for tea.

However, if you set up your constraints on the migration as below, and do not add a validation to the Model, although you will have a fully functioning application, those tests will fail.

While getting Shoulda to work correctly is a not particularly important part of the decision process, it does at least illustrate to anyone that there are fundamental differences between application and database validation. If you didn’t realize it before, welcome to the great Rails Validations Vs Database Schema Constraints Schism, pick your side and prepare to fight.

Hopefully your candidates are fully aware of the differences and can coherently discuss why they made their choice. Which one they pick is less important than their being able to understand what they did and what the opposite choice would mean. This is one of those areas which is a deal breaker for me, if the candidate doesn’t grasp this, its unlikely they have any experience or even theoretical knowledge of scale, and their abilities to make decisions elsewhere will be poor.

So where should those validations and constraints go?

There are pages and pages written on which way is “right.” For what its worth, my preference is to put validation into the application. I love the idea of a database being a dumb storage unit for an application, it leads to consistent, clear code, where all of your logic is maintained in the application space and not shuffled off in a database schema to be forgotten about. As this is a simple prototype its a no brainer decision.,

For real applications, if you’re in an Enterprise where you have multiple languages and tools working together, you’re more likely to have 3 C# guys in Indiana who don’t speak to anyone else and maintain an application vital to accounting, a Rails team and Javascript team at each others throats, where the only agreement between them is that they hate the Java division spread over 3 continents supporting the one application which is responsible for 90% of the company’s income, your 5 most talented engineers have suddenly refused to write in anything other than Erlang, and you are responsible for keeping the entire mess working, update all the legacy applications while simultaneously developing new paradigms and lowering budget 15% per year. So you may need to compromise on what internet theorists prescribe as being the only way to do something. Even in the startup space, you will eventually find your CEO’s wife’s nephew’s friend runs a Visual Basic consultancy and is being brought in to write a couple of tools with direct access to your database. Suddenly you’ve lost 2 days in pointless technical arguments with someone who last updated their technical skills in 2003 and your CEO is printing whitepapers he googled about crosscutting concerns.

So, a simple rule of thumb; if you employ a DBA on your staff, or goodness help you, a DBA team, put validations go into the database schema. If you have dedicated that much of your precious resources to managing and optimizing databases, you are going to be treating them as cross-application components. It is more than likely that your application ecosystem will have grown beyond self-contained units, no matter how hard you work to separate them. If you don’t have a DBA, chances are no one knows how to access all those application databases without going through the frameworks, so you can safely cull anything in the schema and put it in the app code. Oh, and if you have DBAs and don’t have those concerns, fire them and spend the money on something else.

Well, that was some aside, back to Monty Hall.

Rails Callbacks

Lets quickly review our needs

  • We need a game to be set up randomizing a contestant guess and door with the car behind it.
  • We need a mechanism to eliminate all but 2 doors to offer the option to switch.
  • We need to record whether the original guess or switched second guess wins

There are uncountable ways to code the answer, all I would ask is for the candidate to talk through their choices and what could potentially change to make a cleaner solution. I am a fan of Rails callbacks, kicking off the simulation during the create process seems logical, and I think I should split up the functionality between setting up a simulated game and performing the mechanics of elimination to offer a second choice should be in separate methods.

So, I initially solved this with 1 instance method triggered by a before_create callback,  and 2 class methods, with a note to shove the class methods out into a module later, but for now here’s what we have for signatures and return values. I have a hunch at least one controller is going to be fugly, so we have some scope to refactor later.

Rather than copy the code, here are some of the tests:

For play_game there isn’t a great deal of testable code, randomizing some initial values then handing off to another method to handle the door switching logic. So, provided we get all the variables filled in, and we have the right number of doors returned we’ve verified it works. Remember, Monty will remove all but 1 losing door and the initial contestant’s guess, so the doors removed should always equal the initial door count – 2.

The guts of the logic are in do_the_monty_switch. I can forsee two code paths and want to test them both, so these tests will have the same outcome but will test all of the code. Its patently clear that there is scope to streamline here, but for now I figure this will suffice.

And thats about it. Finally, I should test that the instance method correctly kicks off and uses the results to store with the simulation record.

One interesting discussion point is how to perform tests with methods dealing in random numbers. Let me say it now: I do not believe you should change your code to make it testable. There is a real urge to alter the method signatures and pass in seed values so that you can determine the outcome for tests, but I cannot stress enough how much of a waste this is. Again, we’re verging toward another engineering jihad, but my rationale is that I don’t want to test Ruby’s random number generator, or that Rails correctly triggers off a callback, I want to test and write my own code. I can achieve that without asserting every line of every method. I think there is scope for some interesting discussion with your candidates here as you can really examine how effective their tests are, and their decision process of what to test.

Struct, Class Or Hash

Another interesting discussion should revolve around your candidate’s return types. As you can see above, I picked hashes, but you will may see Structs, OStructs and Classes being used. I find all of them to be equally valid with the right reasoning.

I anticipate numerous rounds of changes to the code, so I think its simplest to use hashes until refactoring. I hope I’ve isolated the code which will require changing to just the model methods themselves and the view. I have a couple of tests which will need updating, but it should be pretty painless and is a good test that there is as little coupling between classes and methods within classes as possible.

If your candidates are using more meaty objects, you can discuss where they are located, how Rails’s autoloading works and if you’re really lucky, get into a heated argument on  lib vs app/models !

Rails Best Kept Secret: Hash Stores

When I wrote this, I first coded the model and ran it off the command line. I had some output statements which narrated the game, and on reflection I thought this would be a neat piece of functionality to put in the web app. Now, there are no requirements, so I arbitrarily decided to just do this for the first 50 games of any simulation. If someone is not convinced that the probability is 2/3 : 1/3 after reading 50 games there is no hope for them.

So how to keep this information? I didn’t want to clutter the model with extraneous fields, and I certainly did not want to create another Model with has_many/belongs_to relationships, so I decided to create a log field and stick it in there.

Rails has many hidden or less well known features, somehow turning database Text field into a YAML backed serialization store is one of them. I usually use this feature to store user preferences, save a state machine, or any other kind of information which is highly volatile during development. Setting this up is the kind of one liner which makes my heart sing when I think back to my Java days.

That macro turns the game_log field into a serialized hash store. If you do not specify an object, you can put anything into the field

Rounding Out The Simulation

Ok, we’re just a few files away from being finished on this task. Firstly, lets draw the routes, remember we only want 4 of them, and we won’t get around to index until we get around to the reporting task.

Don’t ask why I preserve the hash rocket in routes.rb. There is no logical reason, I’ve eliminated it from almost all my coding, but for some reason I like the way it looks in routes.

Knowing the routes help define what we’re going to test in the controller. Now, I code controllers to be as thin as possible and I would hope candidates do the same, but if someone can justify what they are doing, and understand how to alter their behavior, that’s fine too. There have been reams written on this and thankfully the Rails community has moved away from fat controllers, but I will sum up my objection in one sentence: I hate writing bloody controller tests, adding logic means adding more tests. When I write more controller tests I am, in the immortal words of Smokey Robinson,  “Really I’m sad, oh sadder than sad.”

I write controller tests to check the routes are working, and where necessary that the model objects are not nil. Aside from that, there should not be anything necessary to test. One of the clues that your controllers are getting putting on weight is when you start testing for more specific items in the session or assigns. We’ll see a great example of this in Part 4 when we get to multi page forms.

Then, the controller is written to be as close to a generated scaffold as possible. Lets look at this controller and bask in it’s nothingness.

Next, we have 2 views; new, show and the form as a partial. Even though I do not have an edit route, I still prefer using partials as its simply good practice. Applications grow over time, coding in a way which does not add any additional development to the initial task, but makes expansion easier later should be the goal for every task.

Finally, I create my layout with a simple bootstrap template. I’m no designer or Human Factors engineer, but I got tired of my raw html or the scaffolded rails css a long time ago. If you haven’t used bootstrap or foundation, invest a small amount of time in it and you will be shocked at what the design impaired can create without expending any time.

Finishing Up

There are a handful of tasks left to make this resemble a real application rather than a hastily finished test. On other tests I’ve reviewed, this is the phase which generally separates candidates’s skills.

For this run through, I want to get integration tests set up, complete a handful of them, and get a basic home page. So we enter the realms of another Rails schism, integration testing. All I am going to say is that I like Rspec, at one point I used it for all my tests, but I’ve simplified my testing where possible, particularly a prototype such as this which only warrants a few tests. So I am using Capybara without Rspec as I believe it fits the scale of what I am testing.

I’ve written enough prototypes to have a standard simple interface with the same layout and ids. So I start with a couple of home page tests which can expand as more functionality is added to the site. I like creating helper methods in integration tests, I will probably refactor these out of the test class eventually.

I create a similar simple integration test for the simulation page. The UI is clean and basic, there isn’t a great deal of testing which can’t be done in the units, and though its tempting to keep writing tests, the result will simply be a duplication of effort. I’ve already tested the model and controller, I’m creating simple sanity tests for the application as a while.

The other problem I am trying to avoid is setting up too many intricate tests tied to css selectors which will change over the evolution of an application. I want to do just enough, and evolve to an appropriate level of testing over time.

Summing Up

Well that took considerably longer to write than it did to code. I finished in about 45 minutes. However, I was sitting comfortably with my feet up, had no pressure, and used gems and tools I have used dozens of times before.

I did add another couple of tasks into the exercise which I am thinking of adding into the requirements, as first written thought, I think 90 minutes would be a fair time to complete. With another 30 minutes discussion after its still a little on the long side, but valuable to get a sense of a candidate’s ability.

What I found most interesting, was that an application with about 30 lines of business logic can spark off so many different decisions and talking points. Obviously, the level of experience a candidate will have should dictate your expectations, but I would hope that from mid level upwards there is recognition and coherent discussion about testing strategy, validation, views/helpers/controllers and return structures.

While this exercise turned out better than I expected for an interview, the more interesting tasks are in the interactive game exercise. We’ll take a look at that next time in Part 3.

The Monty Hall Interview Problem Part 2

Leave a Reply

Your email address will not be published. Required fields are marked *