Fri, 22 February 2013
- Joined by James Roper, committer to the Play 2 Web Framework.
- James started as a Play user, later submitting patches and questions, eventually being hired by Typesafe to work on Play full-time.
- Play 2's philosophy
- Play 2 is a full-stack web framework aimed at making web development simple in both Java and Scala.
- Developers should have everything they need to implement most typical needs, out of the box, without configuration.
- The latest web technologies should be first-class, not bolted on.
- The Play 1 -> Play 2 transition had a few goals
- Change the architecture to make modern web technologies (WebSockets and Comet) first-class citizens.
- Make (idiomatic) Scala a first-class citizen.
- Benefit from Scala's greater opportunities for type checking.
- Why did Play grow in popularity so fast? Probably because it's just the kind of thing Java developers were waiting for after learning about the fast iterations of Rails.
- IO
- Play is built on Netty.
- Play provides an Iteratee library for doing IO with Netty.
- Netty is intentionally not exposed, to prevent the framework from being locked in.
- The body is generally in one of four formats, each of which has a built-in parser:
- URL-encoded form
- Multi-part form
- Json
- XML
- However, structure for custom body parsers/streamers is provided.
- Interop with the Java ecosystem that expencts blocking streams is possible, if not recommended
- You can convert input streams into Enumerators.
- Play is tuned for non-blocking so you need to have enough available threads in the thread pools to handle that blocking.
- Play is designed to make it easy for your app to scale, so blocking is not recommended whenever it can be avoided.
- Routing
- It's completely statically typed and checked at compile time.
- External
routes file compiles to Scala code.
- Reverse router lets you reference routes in a safe way.
- There's also a Javascript router.
- Compilation performance
- This is one of the biggest criticisms of Play, so the developers are focusing a lot of effort on this.
- Avoiding provoking unnecessary compilation from the Scala incremental compiler is a top goal.
- Modularization of routes files and apps helps reduce compilation times.
- Java developers are particularly annoyed because they're used to
javac speeds, not scalac speeds.
- Java vs. Scala
- The Play 1 -> Play 2 rewrite caused some vocal opposition by Java developers.
- There's far more people that are happy with it than are not.
- Play 2 is trying to stay ahead of the game with respect to modern web technologies. Some of the alienated developers may actually be alienated by the changing web landscape.
- Server-side templating
- ScalaTemplates (aka Twirl) are compiled to Scala files.
- Since the template language is just about one special character (the @ symbol), understanding the language and compiling it is easy.
- James is pleased that Twirl is now available as a vanilla sbt plugin for other web frameworks.
- Twirl isn't a DSL for XML or Json, so it's not as DRY as languages like Jade or Scaml.
- You can bring in Scalate yourself if you'd like that. It doesn't type-check template invocation, only the code within a template.
- Binding
- Binding is built right on the case class
apply and unapply methods, plus some validation.
- The Java API uses Spring data binder, which uses reflection.
- Json
- The Play Json library is built on Jackson, adding some wrappers for idiomatic Scala development.
- In 2.0.x Jerkson was also provided for those wanting to bind to and from models using reflection. It wasn't encouraged since Scala can avoid reflection.
- Persistence
- Slick integration is on the roadmap, but can be used as-is right now.
- Slick's string-based interface does something similar to what Anorm does. However, Anorm isn't going to disappear.
- Async
- Responses can be wrapped in an async result if you have reason to think it will take a while to respond.
- The WS API allows you to make web calls, a common reason for slow responses, with the async aspect handled automatically.
- In the case of delays caused by intensive computations, you may want to dispatch the operation to another thread pool that is tuned for it, using Akka's Future type.
- Interestingly, Play suspends the TCP request regardless of whether you use the async function or not. The async function only indicates you want to do asynchronous IO.
- Long-polling is easy, you just use a
Future result. Whenever you redeem the future the result will be sent.
- Comet sockets are h andled through enumerators.
- WebSockets are handled differently, because they aren't HTTP. In Play you return a
Pair[Iteratee, Enumerator] to handle those.
- No pipelining support yet due to a bug in Netty.
- Play's cache doesn't deal with the Thundering Herd Problem. Piping through an actor can potentially solve that problem.
- Criticisms
- Build times are the biggest criticism.
- Alienation from Java developers, especially surrounding the inability to read Play's source code.
- Play 1 features that are not present in Play 2.
- Scaffolding is much desired to generate logical actions that you fill in during development. It doesn't make sense for Anorm, but may make sense during Slick integration.
- Is Play heavyweight? It's not a library, it's a framework, you don't put Play into your project, you put your project into Play. Play 2.1 modularizes things more so less is in your classpath by default.
- What's next?
- Play 2.2 will make the getting started experience even smoother.
- Check out Play Mini.
- Alternate back-ends?
- Play is very interested in Spray, especially spray-io, which will provide deeper integration with Akka, including Akka clustering. Play could switch Netty for Spray today, but the real challenge is how to harness the benefits of Spray's actors while still providing Spray's Iteratees model. Spray tells, Play asks. Investigations continue.
- Servlet 3's async support looks bolted on, as opposed to first-class, so James is concerned that a servlet back-end could be limiting. At best it would be a way to support servlet deployment, not a way to integrate with the whole servlet ecosystem like Atmosphere. Servlet 3.5's async IO may improve things, but maybe not enough.
- Play is looking to support the client-side a lot more, including Javascript framework integrations, Javascript dependency management, exposing server-side models to Javascript, and shared templating.
- Community
- Currently contributing many bug fixes and filling out features.
- To get involved you should find an area you're passionate about and start implementing a feature.
Direct download: 029-james-roper.mp3
Category: interview
-- posted at: 8:28 PM
|
|
Fri, 25 January 2013
- Part 2 of the Interview with Mathias Doenitz on Spray. Part 1 is here.
- spray-routingbridges the API interface directly to your application.
- It does more than just routing, however.
- It creates a structure that determines what logic will be performed on incoming requests.
- Unlike Scalatra, which handles paths as strings, spray-routing uses strong types for path parameters.
- Unlike Play, the set of supported types is open, allowing the developer to bind custom types in routes.
- No pattern matching is used.
- Extremely DRY courtesy of Shapeless.
- Is Spray a full blown web framework?
- Not really. Web GUIs has not been the focus of development so far.
- Session stores, database integration, templating, Javascript integration, ajax support, form support, etc are not provided by Spray. Spray focuses on the API level.
- This may change in the future.
- Who's the most restful?
- Write your application in Javascript, with the back-end being an API only.
- Or, ROCA Style. Read the manifesto!
- With input chunking in the application layer, how do you take advantage of ecosystem libraries that expect a blocking input stream?
- There isn't a really great asynchronous solution. You could buffer chunks and send them to the blocking interface.
- At the moment Spray doesn't have a solution for streaming XML and JSON parsing.
- spray-servletis a small adapter layer that lets you use spray-routing in a servlet container.
- Requires a servlet 3.0 container.
- Streaming isn't availabe because of limitations of the servlet API.
- This is for legacy situations. spray-can is the preferred server.
- The servlet API itself isn't exposed to the developer. It's used internally by Spray.
- Is Spray going to become the foundation for other technologies?
- Anyone can do that, since it's Apache 2.0 licensed.
- Would be open to such a discussion of bringing the Spray team on board to a larger project.
- Reservations
- Is Spray re-inventing the wheel, considering all the effort that has gone into servlet containers and Netty?
- Spray isn't attempting to replace servlets or Netty.
- Spray aims to provide something easy, small, and purely Scala to bridge Akka/Scala-based applications to the rest of the world.
- Is Spray simple? If you're already using Scala/Akka, understand typeclasses and case classes and what not, yes! Spray will seem very familiar.
- Is there some NIH syndrome?
- In the case of spray-json, probably yes, but only because there were not good options at the time the project was started. To address that spray-json may become another implementation of the json4s library and AST.
- In the case of spray-client, probably not. It's desirable to have a client and server in one framework.
- There's room to clean boundaries and tighten modules, just as Akka did in the 1 -> 2 transition.
- Who is using Spray?
- Not completely sure, since users don't always report back.
- Some are listed here.
- Others are confidential.
- Spray isn't going away any time soon!
- Community involvement
- Open to pull requests and contributions.
- Code isn't the only way to contribute. Questions posted to the mailing list are a great way to contribute, as are bug reports.
- Talks at user groups.
- If you do want to contribute code, the best way to get involved is to add a feature you want that isn't there yet.
Direct download: 028-mathias-doenitz-part-2.mp3
Category: interview
-- posted at: 8:23 PM
|
|
Fri, 18 January 2013
- Joined by Mathias Doenitz (aka @sirthias), lead developer of Spray, living in southwest Germany.
- Loves the challenge and learning opportunities that Scala presents. No more daily grind!
- Spray is a collection of libraries for working with TCP/IP and HTTP. It's pure Scala + Akka all the way down to the bare sockets. It is fully asynchronous.
- Why not use servlets?
- Spray can be used on servlets containers if you choose, but most people want to do away with the overhead of servlets and their XML configuration.
- Servlets are server-side only, without a corresponding client library.
- The servlet API's age really shows its age in that it
- Uses mutability.
- Lacks modern function Scala niceness like case classes and typeclasses.
- Lacks streaming + chunked message support.
- Conceals connection management.
- Conceals thread pool management.
- Makes it difficult to monitor/trace your application.
- Why not use Netty? It's great software, but still not Scala all the way down. It's helpful to have the same stack (Scala, Akka, Typesafe Config, Typesafe Console, etc) all the way through, without adapter layers. The Spray primitives are the Akka, Config, etc. primitives.
- Spray is nicely layered. If you need to go lower level, you can always go lower level. There are no black boxes until you reach Java NIO.
- Other Scala web frameworks may be elephants all the way down, but they're not Scala elephants all the way down.
- Core principles of Spray:
- Non-blocking and asynchronous code whenever possible.
- Actor and future-based design. (Also in some cases continuation-passing).
- Highly optimized for high-load environments (think 100,000 open connections).
- Lightweight, with very few dependencies.
- Best practices for actors is still evolving.
- To avoid throttling problems with actors, Spray uses push-back to the network via TCP flow control.
- Spray has ocassionally had problems with untyped-ness of actors.
- Derek Wyatt's forthcoming book will provide valuable information.
- spray-io sits directly on top of Java NIO.
- It is a more faithful representation of packet-based networking. It does not attempt to represent communication as a continuous (blocking) flow of bytes. Instead, spray-io is fully message-based.
- Each actor message contains just they bytes that could be read from the network without blocking.
- There are even cases where Java NIO is too high level, inefficient, or just not thread safe, and Spray is considering some native enhancements.
- spray-can is built on spray-io. It provides a low-level HTTP server & client. In other words, spray-can brings spray-io up from TCP/IP to HTTP.
- The client functionality was originally written just for internal tests.
- The case class (model-only) spray-http module exists so the client and server components can use the same model.
- When the client and server share the same model, certain tasks become really easy, like writing proxies.
- spray-client provides high-level HTTP client functionality. For example, it lets you abstract over the different kinds of connections to a common request-based API.
- Spray has two main levels of runtime monitoring
- Logging directly onto Akka's event bus. Since Spray is Akka all the way down, you can do more with simple logging than is possible in black-box frameworks.
- The Typesafe Console. Spray and Typesafe are working together to make sure Spray gets maximum benefit from the Console.
Part 2 of the interview will be released next week.
Direct download: 027-mathias-doenitz-part-1.mp3
Category: interview
-- posted at: 7:28 PM
|
|
Wed, 2 January 2013
- Joined by Stefan Zeiger from Typesafe, core contributor to Slick, "Scala Language-Integrated Connection Kit".
- Started ScalaQuery as a pet project in spare time.
- Slick involves work in library space as well as in the language itself, e.g., macros. Macros support everything from better error messages to type providers and compilation to SQL.
- Eventually a major expansion on for-comprehensions, called comprehensive comprehensions may be needed by Slick.
- Three front-ends
- Slick has three front-ends: Lifted embedding, direct embedding, and plain SQL. The first and last are the evolution of ScalaQuery. Direct embedding is totally new.
- Lifted embedding, like ScalaQuery, makes you work with wrapped types, like
Rep[Int] instead of Int for an integer column value. Direct embedding lets you work with Int directly.
- The plain SQL front-end is mostly an abstraction on top of JDBC. Provides injection-proof string interpolation.
- Lifted and direct embedding provide a common query optimizer and compiler that gets fed into the JDBC infrastructure.
- Lifted and direct embedding have different strengths. Direct embedding can't completely replace lifted yet, even though it has some compelling advantages.
- Squeryl tries to stay close to SQL, while Slick tries to move away from SQL. It's hard to compose SQL while remaining confident the generated SQL won't fail. Part of the problem is that Slick doesn't see the incoming code, only the computed values.
- NoSQL databases will be supported, starting with MongoDB. Most relational assumptions have already been removed from the master branch of Slick. Table declaration assumption still need to be eliminated, since Mongo works on nested structured documents.
- Slick TestKit is published as its own artifact.
- Comparison to traditional ORMs
- At the moment direct embedding is tied directly to case classes and lifted embedding to tuples. It's difficult for lifted embedding to use case classes without boilerplate.
- Lazy loading is hard to implement efficiently since you don't know the iteration pattern (i.e., the whole query) in advance. Slick does not support it. Slick wants operations to be bounded by the complexity of the query, and not the number of results. Cf. n + 1 problem.
- Slick also avoids magic surrounding mutability, caching, and sessions.
- Future versions
- 1.0 should be out by the end of January.
- Removing dependency on JDBC and SQL in core.
- Adding type providers à la .NET. They will allow your declared schema to be verified. This feature relies on type macros, expected to make Scala 2.11, currently available in Macros Paradise.
- MongoDB support.
- Community role in Slick is currently small, but getting bigger. More input & help is welcome, especially in:
- Driver implementation & optimization.
- Non-JDBC profiles, such as NoSQL databases.
Direct download: 26-stefan-zeiger.mp3
Category: interview
-- posted at: 1:32 AM
|
|
Sat, 15 December 2012
- Joined by Tony Tam, lead developer of Swagger, and Wordnik CEO.
- Swagger is a framework for describing and producing REST APIs.
- The specification component describes the endpoints, the parameter types and counts, input validations, and the resulting data models.
- Once you have the specification:
- swagger-ui can produce a beautiful documentation page for you.
- swagger-codegen can produce a client for you, in many languages
- Unlike WSDL and WADL/2, Swagger is simple, Json-based, and not meant to solve every problem. Some APIs cannot use Swagger.
- Relatively complex model objects can be described, even polymorphic types.
- Even though the specification is in Json, the APIs can choose to accept XML or other formats as input.
- Swagger also produces a service description that can be examined at run-time. This allows more granular access and documentation for different users of the API.
- Some web frameworks integrate Swagger support, allowing you to put the service description in the code, instead of in a separate Json file. Integrations exist for JAX-RS, Play 1, Play 2, Node.js, Sinatra, Symfony, Restler, Django, and Scalatra (which has the "cleanest" integration).
- swagger-codegen uses Mustache templates to produce clients in Scala, Java, Ruby, PHP, Python, Python 3, ActionScript, JavaScript, and Objective-C.
- Generation in the other direction is possible too. Given a Swagger specification, the code generator can write a Node.js, Ruby, or Scalatra API server.
- Chicken or egg? Write the Json API spec and generate a server, or write the server with Swagger support and generate the Json API spec?
- Swagger encourages best practices by focusing on the interface of your API.
- Swagger does not attempt to solve the REST versioning problem. It can facilitate, however, with version paths. Alternatively, a client can examine the run-time description representation and change its behavior according to changes in the API.
- Upcoming: supporting descriptions of authentication, including OAuth.
- Upcoming: supporting descriptions of protocol (e.g., http vs. https).
- Upcoming: supporting descriptions of format, without the need for path prefixes/suffixes (e.g., Json vs. XML).
- swagger-socket allows you to document a wire protocol with full-duplex communication with minimal latency and chatter. Native integration with Scalatra (via Atmosphere) is underway.
- Atmosphere takes the extreme pain out of WebSocket development, transparently solving issues related firewalls, proxies, browser vendors/versions, protocol formats, and HTTP/1.1 fallback.
- Like actor-based systems, sockets are hard to document!
- Community contributions are welcome on GitHub, especially on the code generator (new languages, frameworks, and better implementations).
- #swagger on freenode
Direct download: 25-tony-tam.mp3
Category: interview
-- posted at: 6:48 PM
|
|
Wed, 5 December 2012
- Joined by Ivan Carrero, aka casualjim, core developer of Scalatra and Json4s.
- Ivan developed .NET for 7 years and Ruby for 2 years, including C# MVP. Contributed to ASP.NET Contrib, and MonoRail), IronRuby MVC.
- Moved from Ruby to Scala because of Akka's performance.
- Ivan's comments on type erasure gets this episode an "explicit" tag on iTunes! But Scala 2.10 will fix most of his language gripes.
- Scalatra is a tiny web framework inspired by Sinatra. Core philosophy: everything is extensible by default; you should never be worked into a corner.
- Sinatra is a thin layer of Rack (Ruby web server interface, think Servlets) and HTTP. Sinatra stays out of your way. Not opinionated like Rails.
- Sinatra lacks model, validation, etc support. Padrino complements Sinatra, especially for developers not writing an API service.
- Scalatra's core is a direct port of Sinatra, except for the route-matching strategy. Once you bring in modules outside core, Sinatra is Padrino and more.
- The biggest competitors are Play & Lift. Play is inspired by Rails and is likewise opinionated.
- Finagle has beautiful routes extraction.
- The Sinatra-style route matching lets you understand a web app without jumping around too much in the files.
- Scalatra is designed to be easy to use, "zero to hero". Cleverness is avoided. "Hello World" is 4 lines.
- Route patterns are the arguments to the blocks they execute.
- Templating is done with Scalate (How is that pronounced?!), which gives you 4 different templating languages to choose from (2 of which are xml/html DSLs). Integrations exist for others too.
- Separate support for Jackson and Lift-Json lead to Json4s, one AST to rule them all! Currently Jackson and Lift implementations exist, and discussions are underway with Spray (a Parboiled parser).
- Json4s also provides monadic ways of manipulating the AST.
- Scalatra accomplished extensibility mostly through the use of stackable modifications, callbacks, and inheritance. Mix-in order generally does not matter.
- Scalatra comes with Sentry, an auth module inspired by Warden. It can protect routes or entire servlets.
- Some kinds of re-usability, in absence of a framework-wide opinions, is a rendering pipeline.
- Scalatra takes advantage of Servlet 3.0's support for asynchronous requests and responses, hooking them to Akka futures. Suspending requests allows you to free the thread while other requests are handled.
- Netty requires you to parse chunks of a request, where a Servlet container gives you that for free. Jetty is mostly as fast as Netty until you get to web sockets and persistent connections.
- Jeanfrancois Arcand (of Atmosphere and async-http-client fame) and Ivan are looking to bring Scalatra to Netty. Currently the Netty branch doesn't perform as well as the Jetty version.
- Scalatra commands provide validation. Unlike most frameworks, it validates the request headers and bodies directly, not waiting for de-serialization into a model. Inspired by the CQRS pattern. Ad-hoc typeclasses can convert them to model classes.
- Swagger is a service description language, but nothing like WSDL! It's all about the metadata … once you have it, you can examine it in code and make decisions based on it.
- Swagger integration is provided in the form of method calls that are fed to the Swagger documentation and code generator.
- Integration with Atmosphere provides a servlet container agnostic way to do web sockets, comets, server-side events, etc. It can fall back to long polling for browsers without a socket implementation.
- Upcoming: etag-based caching with pluggable back-ends.
- Upcoming: metrics (from Coda Hale's Metrics library).
- Upcoming: SwaggerSocket support.
- Upcoming: making it more difficult to accidentally close over scala.util.DynamicVariable request/response objects in an Akka future.
- Scalatra is used by BBC, the British Government, IGN, Jelastic, LinkedIn, the Guardian, and others.
- A Scalatra book is coming out in 2013, published by Manning.
- Learn more on #scalatra on freenode. It's easy to contribute to Scalatra.
Direct download: 24-ivan-porto-carrero.mp3
Category: interview
-- posted at: 12:01 AM
|
|
Mon, 5 November 2012
- Joined by Eric Torreborre, author of specs2.
- Worked in financial computing; now works for NICTA.
- Currently developing scoobi, a Scala productivity framework for Hadoop.
- UML, specifications evangelist.
- First wrote specs, to learn Scala and bring test-driven development to Scala. Later rewrote specs as specs2.
- UML is closely tied to object-oriented languages, so its limited in its ability to model higher-order functions in Scala.
- specs2 philosophy document.
- "Avoiding side-effects" may be a better term than "immutability" for describing the changes in specs2.
- specs relied on mutating variables for establishing the context of a test (i.e., setup and take down).
- specs2 also avoids throwing exceptions to indicate failure.
- Before/after test isolation in the style of JUnit can be expensive, and isn't appropriate for a value-oriented test framework.
- In specs2, tests have a meaningful type depending on the expect result, not
Unit/void.
- specs2 does maintain a "mutable" exception-throwing option to provide a migration path from specs. This introduces just one side-effect: the building of the specification itself.
- specs2 immutable tests chain examples (tests) together in a pure function:
class MySpec extends Specification { def is = example1 ^ example2 ^ example3 ^ end }
Direct download: 23-eric-torreborre.mp3
Category: interview
-- posted at: 2:16 PM
|
|
Mon, 22 October 2012
- Joined by Derek Wyatt, author of Akka Concurrency, now in pre-print from Artima.
- Akka framework
- Multi-core world
- Shared state concurrency difficulties: Heisenbugs, core dumps, deadlock
- Message base concurrency: Unix pipelines, RabbitMQ, AMQP standard, REST
- Actor-based concurrency and Akka actors
- Futures and promises, Akka Futures, Java Future and Scalaz Promise.
- When should you use futures, and when should you use actors? Set-order operations like matrix multiplication favor futures.
- Akka requires indirect references to actors and organization into actor systems.
- Akka's fault tolerance.
- Typed vs. untyped actors.
- Futures compose better than actors, and without side-effects.
- How do you test actor systems?
- Dataflow concrrency is only available in the Scala API due to the use of Scala's Continuations.
- Agents, famous in Clojure, also in Akka. Agents are just a bit like java.util.concurrent.atomic classes made far more general.
- Antipatterns: not programming in the spirit of the actors paradigm!
- Throttling messages in Akka: blog post and documentation.
- Use Akka from the beginning if you may need it later. Bolting it on to a finished product is difficult.
Direct download: 022-derek-wyatt.mp3
Category: interview
-- posted at: 7:26 PM
|
|
Mon, 8 October 2012
- Stairway Book
- ScalaTest
- Scala 2.10's Macros will improve error messages for assertions.
- ScalaTest 2.0 (see M4 Release Notes)
- Tool Integration
- ScalaTest DSL for Selenium.
- Selenium has DSLs for other languages, like FluentLenium in Java.
- Tag class to tag all tests in the class.
- Cancelled tests to indicate a pre-condition of the test has failed.
- Better test parallelism.
- Support for testing concurrent code.
- specs2
- specs2 emphasizes immutability over specs.
- Read these! ScalaTest philosophy & design, specs2 philosophy and design.
- Power assertions: expecty, spock
- Some featues of specs2, like one assertion per test, typed test results, and html reporting are coming in future versions of ScalaTest.
- No test balkanization!
- What features are planned for ScalaTest after version 2.0?
- Native XML file format
- Assertion & matcher enhancements
- sbt integration come after 2.0 depending on when sbt 0.13 is released.
- HTML specification output.
Direct download: 021-bill-venners.mp3
Category: interview
-- posted at: 5:12 PM
|
|
Sun, 23 September 2012
Direct download: 020-benjamin-pierce.mp3
Category: general
-- posted at: 11:58 PM
|
|