Swiftkiq - job queue system on Swift

Inspiration

Swift can be done all your tasks. - Swiftがすべての仕事を終わらせてくれる
Run swift on your powerful servers as you like. - 最強のマシンの上で好きなだけSwiftを働かせよう

What it does

Swiftkiq is the framework for job queue system.
Work your swift code on distributed environment as job queue system
- manage job's retry flow
- manage threads
- manage process as daemon

Samples

here is runnable example https://github.com/ainame/Swiftkiq/tree/master/Examples

worker

  import Swiftkiq

  class EchoWorker: Worker {
      struct Args: Argument {
          public var message: String
          public func toDictionary() -> [String: Any] {
              return ["message": message]
          }

          static func from(_ dictionary: Dictionary<String, Any>) -> Args {
              return Args(message: dictionary["message"] as! String)
          }
      }
      var processorId: Int?
      var jid: String?
      var queue: Queue?
      var retry: Int?

      required init() {}

      func perform(_ args: Args) throws {
          print(args.message)
      }
  }

run script

  import Foundation
  import Swiftkiq

  let router = Router()
  let options = LaunchOptions(
      concurrency: 25,
      queues: [Queue(rawValue: "default"), Queue(rawValue: "other")],
      router: router,
      daemonize: false
  )

  let launcher = Launcher(options: options)
  launcher.run()
  while true {
      sleep(1)
  }

How I built it

port Sidekiq on Ruby to Swift
build demo application

demo application is here https://github.com/radioboo/Scraper

Challenges I ran into

Different from ruby's implementation

type safe arguments
can run native built code
can use multi core cpu
- ruby is stricted by GIL(global interpreter lock)

Swift's problem

Dynamic dispatching workers from string
- use krzysztofzablocki/Sourcery to generate routing table for workers
Can't use thread local storage with GCD
- make instance cache be thread safety
There is no perfect Redis client.
- Patch to Kitsure-redis