Summary: Panel Avdi Grimm (twitter github blog book) Charles Max Wood (twitter github Teach Me To Code) David Brady (blog twitter github ADDcasts) James Edward Gray (blog twitter github) Josh Susser (twitter github blog) Discussed in this podcast: Definition of Queuing and Background Processes: Queuing is about messaging. Typically first in first out (FIFO) Background Processes are processes that pull messages off the queue and do things later. Systems that people have used Beanstalk Resque cron RabbitMQ bunny ZeroMQ BackgroundRB AMQP Delayed Job Queue Classic Poor man’s queue or database backed queue What do you look for in your queue technology? How it takes failure Introspection It depends on what you’re doing Regular tasks vs. Immediate tasks One worker vs several workers on several servers Dave’s beanstalk utilities Beanstalk RailsCast Hybrid approaches Google’s Geocoding API - Rate-limiting with cron Amazon’s FPS - polling amazon to get updates on payments Best Practices Structure jobs so they are simple input/output Isolate your jobs as much as possible from the database Decouple your application from your queue - Message Passing Distributed == Not Dependent - If the job dies and you are broken, you’re designed wrong Idempotence on the job Messaging Queue == State Machine Logging/Emailing when something is unprocessed for too long What do you send to the background? Payment Processing Emails PDF Generation Billing reconciliation Geocoding External Web Services Collate events into an aggregate event Farming out multiple jobs to multiple queues Separate unusual resources onto other servers Picks Service-Oriented Design with Ruby and Rails (Addison-Wesley Professional Ruby Series) (Dave) Enchantment: The Art of Changing Hearts, Minds, and Actions (Dave) If you gaze into nil, nil gazes also into you (Avdi) hacker monthly (Avdi) rubythere.com (Josh) code for america (Josh) Philosophy Gym (James) Philosophy Bites (James) Ask a Ninja (Chuck) Ruby 1.9 (Chuck) RVM (Chuck)
