RailsConf: Nick's Highlights from Windy City Rails

I attended and spoke at Windy City Rails. I tried to take more notes this time. Out of 6 talks an lightning talks, I have ~500 lines of notes. Enjoy!.

Jake Scrugges - Metrics Based Refactoring: What To Do With Your Code Metrics

• Metric-Fu is a ruby metric bundle
• rake metrics:all
  => generates a master report of a bunch of metrics
• There are a lot of metrics, where do you start?
• Everyone wants to refactor, but where do you start?
• Flay
  • Analyzes code duplication
  • Finds violations of DRY
  • Solutions: Extract Method, Modules, etc.
  • Single Responsibility Principle
• Churn
  • Looks through your commits to see what stuff changes the most
  • Files that change the most are "God Objects" - stuff that does everything
  • Any time anything changes, the God Object has to change
  • Bad smell, should be smaller objects with greater focus
  • Tells you where your most volatile code is
• Rcov
  • Main reason metric-fu blows up
  • Problem is, metric-fu is mostly static analysis, but rcov is running the code
  • Rcov runs the test suite, which is sometimes hard to get working
  • Rcov makes sure your coverage is good where it needs to be (most complicated and volatile code)
  • Writing tests for the sake of coverage is harmful
  • Write tests when you are in the code and you understand it (not a week later, "let's write some tests!")
• Flog and Saikuro
  • Both are complexity management
  • Saikuro (pronounced cyclo as in cyclomatic complexity) measures cyclomatic complexity
  • Saikuro = number of paths through the code, and cyclomatic complexity is exponential in growth
  • Flow looks at branching, but uses ABC
  • Assignments
  • Branches
  • Calls
  • Flog generates a score based on ABC
  • < 20 is good, 20-40 is a grey area, 40-60 is a warning, 60+ is FIX IT!
  • How do you fix it?
  • Extract method
  • Missing object (need a new class to handle this behavior)
  • If that doesn't work, re-architect
  • Seriously, you don't need complicated methods in any app
  • Director method: calls a couple of sub methods, doesn't do much actual work
  • Take care not to over-factor. If you are a guest in the code, just clean it
    as necessary. You don't need to fully re-organize.
• Reek, Roodi, and RailsBestPractices
  • Reek: if it could possibly be a problem, I'll tell you
  • Roodi: only if I'm really sure, I'll tell you
  • RailsBestPractices: just rails stuff
  • All parse the code and look for design problems
  • Very ambitious! Code trying to understand what you're trying to do
  • Awesome because you can look up the definition of the problem and see how to fix it.
  • Trouble because it's a machine trying to identify human problems, which is difficult
• Continuous Integration
  • Metric-fu on your code
  • Most CI's allow you to have artifacts for a build
  • J.S. runs his nightly at midnight
• Pay attention to trends over time
  • Is your refactoring making things better?
  • Did the last feature skyrocket flay scores?
  • How is the influx of new developers affecting the code quality
  • Did the highly paid consultants produce good work?
• Take metrics as advice (not law)
  • It is a tool, use it intelligently (like a GPS system)
  • Don't let a manager get in charge of this
  • Up to you and your team to establish conventions and stick to them
• Coming soon
  • Fail the build on bad metrics
  • Resisted for a while because there may be false negatives
  • Failing a build is a serious incident, needs to mean something
  • Metrics are not as definitive as tests
• Coming later (and/or soonish)
  • Meta metrics
  • Imagine a report that flagged a method as having high churn, poor coverage, and high complexity
  • This method is trouble waiting to happen
  • No so easy to write this feature
  • J.S. needs help!
  • Need to re-architect the system to understand classes and methods, not just lines of code
• Questions
  • Can you use other source control?
  • I think so, check out the churn gem
  • How is the performance?
  • Not great, but you only have to run it once a day
  • If you don't have test coverage, now what?
  • start w/ rehacktoring (refactoring w/o tests)
  • do that until you understand it enough to write tests
  • write tests!
  • What about front-end heavy stuff? Javascript?
  • No idea! These are all ruby tools. Maybe there are JS tools out there?
  • You should test your javascript. Sapphire is the "new hot thing".

John McCaffrey - Analyzing and Improving the Performance of your Rails Application

• http://railsperformance.blogspot.com
• O(n) queries is bad
• Code he refactored went from 2811 to 17 queries per action
• Leverage: small amounts of work for large amounts of value
• Delay directly correlates w/ loss of revenue
  • Bing: 1s = 2.8% decrease, 2s = 4.3%
  • Google: users that had a delay for a short period of time, actually searched less even after the delay was removed.
  • Yahoo: added a 400ms delay and users bailed before the pages even loaded
  • Shopzilla: reduced delay from 6s to 1.2s.
  • Conversion rate rate up 7-12%
  • Saved infrastructure costs by 50%
  • Release cost decreased
• Fred Wilson (VC): Speed is #1 of the 10 golden principles of successful web apps
• So where to begin?
• Anti-patterns:
  • "Scalability can be sprinkled on later"
  • Guessing, not testing
  • Ad-hoc, unstructured
• "Premature optimization is the root of all evil" - Donald Knuth
• "If you can not measure it, you can not improve it" - Lord Kelvin
• Measure
• Repeatable tests
• Isolate your changes
• (Scientific method!)
• Visibility is important for working on a team
• Response time = latency
• Requests per second = throughput
• Load, utilization, scalability, throughput, concurrency, capacity
• Performance != Scalability
  • Performance = response time
  • Scalability = ability to grow to more requests
  • Adding more workers will not improve latency
• Twitter: dynamic html = 250ms, but 3369ms is static assets
  • Don't waste your time on the 250ms, go for the big slow chunks
• Walmart: 3 redirects, 388ms dynamic, 4580ms static content
• Use YSlow
  • Do less work
  • Distribute content
  • Cache
  • Compress
  • Fight queued work
• Google page speed
  • similar to yslow
  • includes paint events
  • separates out ads and trackers from your site
  • "how fast it feels"
  • If the web is faster, they make more money
• http://www.showslow.com/ tracks your site (or competitors, will check every 24h)
• http://webpagetest.com/ runs page speed and yslow for you, and show progressive rendering
• http://gtmetrix.org
• http://zoompf.com
• http://loadimpact.com
• http://gomez.com
• See Greg Pollack's Scaling Rails series
• Majority of the Alexa top 1,000 sites don't do even the easiest things
  • 42% don't gzip
  • 44% >2 css files
  • 56% serve css from a cookied domain
  • 62% don't minify
• Gzip is the simplest and smartest thing you can do
• Minify js
  • jsmin, yuicompressor, sprockets
  • Asset packager
  • javascript_include_tag :cache => "cache/all"
• Sprited images
• Spriteme.org
• JqueryUI gives you sprited images
• Image optimization
  • Smush.it
• Expires and browser caching
• Set a far-future expire to tell the browser it can cache it
• Combined with a query string lets you version a file
• Rails Caching: make the static parts of your dynamic content cached
• railslab.newrelic.com
• request-log-analyzer
• Rack::Bug
• NewRelic
• Scout
• Get on the heroku mailing list / blog to learn about scaling issues
• Apache bench
• httperf
• jmeter
• Database issues
  • Bad queries
  • Not utilizing explain
  • Inadequate indexes
  • N+1 queries
  • Selecting more data than needed
  • Inconsistent queries for the same data
  • http://github.com/dsboulder/query_reviewer/
• http://github.com/eladmeidar/rails_indexes
  • rake tasks to find missing indexes
  • columns that need to be sorted
  • lookup fields
  • columns used in a group-by
• http://github.com/samdanavia/ambitions_query_indexer haven't tried it yet, check it out
• http://github.com/sdsykes/slim_scrooge Instruments your code and looks for bad "select *"
• Aman Gupta
• Joe Damato
• http://timetobleed.com/
• http://memprof.com/
• yajl-ruby for C++ extension json parsing
• Ruby 1.9 for massive performance

Kevin Gisi - It's time to repay your debt

• We're leveraging an awesome framework and language, and it's time to give back
• Releasing gems is easy
• (Showing some code to refactor out into a gem)
• Ruby community has an incredible (and overwhelming) amount of choice
• Choice is good and important
• There exist incomplete and unstable solutions
• You should release a "product", not an incomplete solution
  • Aesthetics
  • Utility
• Agile: Do the bare minimum to get something delivered
  • (Nick: isn't it to embrace change?)
• Delivered: coded, deployed, supported, tested, extensible, usable by the client
• Treat open source libraries and client deliverables
• The Lazy programmer's manifesto
  • I want to write code once
  • I want to be able to read code w/o docs
  • I want to be able to rely on libraries
  • I wants to use adaptable solutions for numerous problems
  • I want to always be working on something new
• Idea filtering
  • To share
  • business value
  • practical use
  • To learn
  • no requirements
  • don't post it publicly (community does not value your stupid web framework)
• Testing
  • We all say we do it (it's time to start)
  • RSpec and Cucumber are easy to set up
  • Design verification tool
• (really turning into a rant at this point)
• Extensible
  • Write clean code (don't post spikes)
  • Peer review
  • Follow the Unix philosophy: do one thing really well
  • Leverage other gems
• Documentation
  • Who's looking at your code?
  • RDoc, YARD, etc
  • Getting Started
  • Blog post!
• Supported
  • If you can't maintain your app, don't post it
• Be Honest
  • Hand over the reigns if you can't support it
  • Be a net positive
  • Don't announce things that aren't done
• If you have to
  • Put it in a limited venue
  • Add a disclaimer
• The internet doesn't filter bad ideas very well
  • It does make things hard to do if they don't work well
  • Having enough eyes
• "Ruby treats you like a grown up programmer" - Matz
• What happens when grown-ups have too much fun?
• How to be a good parent
  • Avoid sensory overload
  • Sometimes spoon feed
  • Remember you're used to the status quo
  • Be a mentor http://railsmentors.org
  • Provide solutions to problems, not blocks of code
• (I wish this was a "how to do a good job" and not "you suck, stop sucking". I think he wants to be the king of the rubygems app store)

Lightning Talks

Micah Martin - Limelight

• Tool to build GUIs in ruby
• Built a gui calculator in 5 minutes

Sean Scofield - The Awesome Power of Rails Engines

• Engines make it easy for frameworks to be built on top of rails
• Engine is a complete application that can provide the stuff rails apps provide
• app directory is auto loaded
• You can supply models, views, controllers in your engine
• You can override locale by dropping it in app
• Engines extending engines, for example Spree
• Bundler ties it all together

Mike Buselli - Brush::Pipeline

• Running external OS commands
• sys
  • lower level, more convenient for specific takss
  • chaining commands
• pipeline
  • better at chaining commands
  • pipeline ['head', 'README'], ['tail', '-n2']
  • Each part of the pipeline can run in different directories
  • stderr and stdout redirection
  • argv[0] renaming
  • restrict access to open files
  • Works great on POSIX including cygwin

Chris Hallendy and Nick Lewis - Realtime Apps

• Demo of node.js <-> Redis <-> Rails

Joe Fiorini (sp?) - Web Katas

• Problem: web developers don't know http
• Rails is big and awesome, but it's hard to troubleshoot
• It allows us to ignore how http works.
• Code Katas http://bit.ly/codekatas
• No-framework webapp development (Rack) to learn how http cookies work
• More Katas
  • Auth token
  • See Other response status
  • Move permanently
  • Session state handling
  • http://bit.ly/webkatas

Jim Remsik - Improv: Comedy to Coding

• Programming is a people problem
• You write code for other people
• We use ruby because it is expressive and easy for others to read and maintain
• Improv after-hours at hashrock
• Yes-and, drop your preconcieved notions at the door
• Support your team

Malcolm Arnold - Ruby Nuby

• Brand new social movement dedicated to teaching ruby on rails to disadvantaged youths
• Maximize social good by spreading ruby

Praveen Alavilli - Monetizing your apps

• "chase your dream, money will follow"
• Still a mystery to a lot of people how the money follows
• Indirect
  • advertising
  • offers
  • referalls
• Direct
  • e/m-commerce
  • freemium
  • causium (donate money to a cause)
  • pay as you use
  • free to use pay for service
  • premium content
  • digital goods
  • virtual currency
  • subscriptions
• Challenge: apply these models to your use cases
• Gateway (i.e. Payflow, Authorize.net)
• Checkout (Google, PayPal, Amazon)
• Platform (PayPal, Amazon)
• Also services on top (Chargify, Spreedly, Freshbooks)
• Choose one that works the best
• Your users need to be able to pay somehow, should not be hard to pay
• PCI compliance for charging directly
• secure and privacy enabling
• Adaptive Payments API

Ryan Singer - Weaving Design and Development

• UI Design and Programming, working together
• (telling a story about working with a startup and doing design for them)
• Designer says "let's change this button" while pairing with dev, dev makes the change instantly
• XP is focused on delivery
• In XP, feedback loops are not as important
• Feedback can be difficult if you are purely focused on delivery
• Shared code base, code runs on designers computers
• Avoid the designer w/ folders full of PSDs and static content
  • Becomes a spec, then the dev has to implement it
• Static files are like spikes, for exploration and experimentation
• Actual changes need to keep up with the codebase
• At 37Signals, no sync problem, because the designers are working in the code base
• HAML and Mustache are out, because designers can't deal with them
• Templates with logic
• Avoid helpers writing HTML and presenters writing html
• Start together, don't bring the designers in late, or the developers in late
• Architecture before design leaves UX to too late in the process
• Designers starting everything before programmers start, it may be impossible or impractical to code
• P&D decide on domain language and basic model
• P stub models, controllers, routes, and templates
• D few days lead on designing and building templates
• Simultaneous
  • P design and build models, controllers, routes, helpers, dynamic templates
  • D design and build templates and helpers
  • Repeat this process continuously
• Share the codebase
• All views are in templates w/ logic (ERB or similar, not HAML)
• Start together for domain language and models for communication
• constantly committing to the same codebase
• Train designers in git and ruby and other things
• Designers are motivated to learn so they can make changes directly: it gives them more power

Nick Gauthier - Grease your Suite

Les Hill and Jim Remsik - Sustainably Awesome

• Comfortable and productive environment is crucial
• sustainable pace is important. 40h weeks.
• cargo culting
• Pet friendly
• The point is not to do what we do, but to recognize our goals. Don't cargo cult these tips!
• Casual
• No Micromanagement
• Don't penny pinch
• Keep a budget and spend it wisely
• $400 per week grocery budget
• No death marches
• No meetings (except daily standup and project standup and monthly company status meeting)
• No hierarchy
• No future-proofing (software and organizational practices)
• What do you do when a developer rm -rfs?
  • No sudo for devs?
  • 3 manager approval before running a script?
  • Develop a backup solution and a common machine image?
• "Process is an embedded reaction to prior stupidity" - Clay Shirky
• A project came along that needed work done ASAP
• Crafted an MVP and built it without changing their process
• The MVP did not overlap with their mockups and specs
• Started with story cards and team boards
• Moved to pivotal tracker
• Tim Pope had a suggestion about how to work stories in pivotal that worked really well
• Always open to try new things and experiment with new processes
• Enjoy your work
• Don't hire people you don't like, or without passion
• Hire smart people that get things done
• Must be a great cultural fit
• Hiring
  • phone screening
  • week long "test drive"
  • gems you use are important
  • editors you use are important
  • test driven (and test-first) style
  • hang out after hours to check for a cultural fit
• "If I'm not saying 'hell yeah!' about something, then I say no" - Derek Sivers
• Firing: due to lack of communication or mistake, probably on both sides
• Fire fast, but with compassion
• Approach building a team the same way you build software

Yehuda Katz - Rails 3

• Rails core family is getting bigger
• Rails 2 was very intertwined
• Broke things up cleanly
• Instead of Railties being a big glue layer w/ initialization stack, it became a small part or Rails 3 and specifies the framework for applications and engines
• Engines are special plugins now that inherit from Railties
• Applications inherit from Engines
• Railtie lets you add features to rails
• Railties provide a series of Hooks
• Inheriting from Railtie doesn't do anything, but it has lots of functionality
• Railtie lets you configure the application, such as pull in generators
• Rake Tasks are another Railtie Hook
  • Extra stuff you want to have happen in rake, and not on boot
    rake_tasks do
    load "my_stuff/my_tasks.rake"
    end
• before_configuration hook = right inside Rails::Application, before user stuff happens, but you know about rails root and stuff.
• config.to_prepare : in dev, run every time, in production, run once (i.e. compass template compilation)
• Framework loading: ActiveSupport.on_load = i.e. when action mailer loads, monkey patch in some code. Then you don't have to try to detect ActionMailer, you just say, when it loads do stuff.
• Don't do if(defined) because it depends on order
• Decouple via Instrumentation
• ActiveSupport::Notifications.instrument("receive.action_mailer") do |payload|; #callback; end
• Worth Noting: we worked really hard making things mostly backwards compatible
• The old way may appear to work, even if it doesn't
• AbstractController is decoupled from ActionController
• ActionDispatch = Rack++
• ActiveModel lets other persistence engines connect to ActionController. Only need to support ~5 methods
• Tight Integration without Coupling
• Integration first, decouple later
• Migrating:
  1. Install Rails 2.3.9.
  2. bundler
  3. rails_xss
  4. Deprecations - get the count to 0
  5. Upgrade
  6. Deprecations again!
• Helpers should call partials, not generate html
• Rails 3.1: We're not burned out!
  • Engines
  • @drogus did all the engine work
  • Engine Migrations
  • Static assets (w/o copying). Leave it in the engine, and Rack::Static will find it
  • Self-contained engine
  • Built-in http caching, to get away from filesystem caching
  • fresh_when? calculates an etag from the post and returns 304 if not fresh
  • caching at the browser layer
  • etag can be based on an AR object
  • If you have Varnish, you can turn off Rack::Cache and fresh_when will be built in
  • This also means that when you turn on Akamai, you already have all the code that makes it work well
  • Enables Edge-Cached SCSS
  • Rack::Cache is like Memcache, but uses HTTP semantics
  • Assets
  • ERB == SCSS (I have content that needs to be processed before it hits the browser)
  • Solution: build in compilation pipeline
  • app.css.scss (like app.html.erb)
  • posts/index.css.scss
  • Treats CSS the same as HTML for templating
  • SCSS template handler => http response => rack::cache
  • It just works w/ varnish and the like
  • Spriting
  • Compass and lemonade are merging w/ a new sprite api
  • Put all your images in images/sprites/ 1, 2, 3, 4, .png
  • Lets you include all the images as a sprite
  • Gives you a bunch of css classes to use the sprites
  • SCSS can take CSS, and then add stuff in
  • You don't need imagemagick, there is a pure ruby PNG spriting library
  • AutoFlush - it's cool, read his blog, running out of time
  • flush content to the browser as it is rendering
  • Tradeoffs you have to deal with, like exceptions
  • Browser can be working on your CSS while you are doing SQL queries
  • Keep content_for at the top of the template
  • Exceptron
  • Stolen from merb
  • ExceptionsController to render and take actions when exceptions happen
  • Performance
  • We did pretty well in 3.0, definitely some trouble spots
  • More mechanisms for performance
  • Big Arel rewrite for 3.1
  • Possible to get to 2x faster over 3.0

Want to learn more about RailsConf, check out our other articles:

The Strength of the Rails Community, as seen at RailsConf

Being Social At RailsConf Without Talking Tech: Possible?

Links from RailsConf 2013