内容简介:Async HTTP— 2020-02-25Today
Async HTTP
— 2020-02-25
Today Friedel Ziegelmayer (Protocol Labs), Ryan Levick (Microsoft), and myself would like to introduce a new set of HTTP libraries to make writing encrypted, async http/1.1 servers and clients easy and quick:
-
async-h1
– A streaming HTTP/1.1 client and server protocol implementation. -
http-types
– Reusable http types extracted from the HTTP server and client frameworks: Tide and Surf . -
async-native-tls
– A streaming TLS client and server implementation.
None of these projects are official projects by any one company.
This library builds on the excellent work done in
native-tls
.
With these libraries writing a streaming, encrypted HTTP client takes about 15 lines of code:
The TCP stream, HTTP stream, and TLS stream each expect a different subset of a URL. Most of the logic in this example is about constructing the right argument for each stream. We have an open issue that if implemented should make this even easier.
use async_std::io::prelude::*; use async_std::net::TcpStream; use http_types::{Method, Request, Url}; #[async_std::main] async fn main() -> http_types::Result<()> { // open a tcp connection to a host let stream = TcpStream::connect("127.0.0.1:8080").await?; let peer_addr = stream.peer_addr()?; // create a request let url = Url::parse(&format!("https://{}/foo", peer_addr))?; let req = Request::new(Method::Get, url); // encrypt the tls connection let host = req.url().host_str()?; let stream = async_native_tls::connect(host, stream).await?; // send the request over the encrypted connection let res = async_h1::connect(stream, req).await?; println!("{:?}", res); Ok(()) }
And an async HTTP server is done in less than 30 lines, with much of the code being imports,async loops, print statements, and constructing the right URL parameters:
use async_std::net::{TcpStream, TcpListener}; use async_std::prelude::*; use async_std::task; use http_types::{Response, StatusCode}; #[async_std::main] async fn main() -> http_types::Result<()> { // Open up a TCP connection and create a URL. let listener = TcpListener::bind(("127.0.0.1", 8080)).await?; let addr = format!("http://{}", listener.local_addr()?); println!("listening on {}", addr); // For each incoming TCP connection, spawn a task and call `accept`. let mut incoming = listener.incoming(); while let Some(stream) = incoming.next().await { let stream = stream?; let addr = addr.clone(); task::spawn(async { if let Err(err) = accept(addr, stream).await { eprintln!("{}", err); } }); } Ok(()) } // Take a TCP stream, and convert it into sequential HTTP request / response pairs. async fn accept(addr: String, stream: TcpStream) -> http_types::Result<()> { println!("starting new connection from {}", stream.peer_addr()?); async_h1::server::accept(&addr, stream.clone(), |_req| async move { let mut res = Response::new(StatusCode::Ok); res.insert_header("Content-Type", "text/plain")?; res.set_body("Hello"); Ok(res) }) .await?; Ok(()) }
Creating an encrypted version of this would be only a few extra lines: load a
certificate, and wrap the accept
stream:
use async_std::prelude::*; use async_std::net::TcpListener; use async_std::fs::File; let key = File::open("identity.pfx").await?; let pass = "<password>"; let listener = TcpListener::bind("127.0.0.1:8080").await?; let (stream, _) = listener.accept().await?; let stream = async_native_tls::accept(key, pass, stream).await?; // Continue handling the stream here
Streams
As you may have noticed, the libraries we've written are centered aroundstreams. This is because streams in Rust provide an incredible degree of composability . For example, using surf to copy the body an HTTP request to a file is a combination of "make an HTTP request", and "copy to a file":
let req = surf::get("https://example.com").await?; io::copy(req, File::open("example.txt")).await?;
With async-h1
and async-native-tls
we wanted to not only make this
possible at the framework layer; we wanted this to extend to the protocol
layer as well. Because we believe that if it becomes easier to combine parts
of a stack, it becomes easier for the ecosystem to make progress on the stack
as a whole.
Putting it concretely, if you want to run async-h1
over a
UnixStream
to say, communicate with a local
daemon
, all you have to do is
replace the TcpStream
with a UnixStream
and you're good to go. There's no
need to start from scratch or fork existing projects.
Shared abstractions
As we mentioned at the start: the http-types
crate has been extracted from
the Tide and Surf frameworks. Prior to this we were using the
hyperium/http
crate, which provides
abstractions for several HTTP idioms, but for example doesn't provide an HTTP
body, cookies, mime types, and doesn't implement the url
standard
. These are all fine choices with
distinct tradeoffs. But for us
we were finding that we had a lot of
duplicate code between Tide and Surf, which was becoming a maintenance
burden.
If the past is any indication, it's likely someone will bring up the "You're splitting the ecosystem" meme when discussing this post. This is an argument to authority: rather than explore how things could be improved, we should relinquish to the status quo. But this conveniently fails to acknowledge that writing and sharing code in the public domain is collaboration. Our ideas are now available to try out and improve upon with no restrictions other than attribution. Sharing findings in the public domain is not a divisive act. Fanning controversy using "us vs them" rhetoric however, is.
So instead we've opted to create http-types
, a shared abstraction that
covers the full range of HTTP idioms, and provides a streaming
body. We've found that this has made it easier to create a rich set of HTTP
abstractions for us, to the point where we suspect both Tide and Surf's types
will be minimal wrappers around http-types
and
http-service
and
http-client
respectively. Only bringing in framework-specific additions
such as middleware and routing.
Error Handling
Additionally http-types
provides a status-code aware Error
type, making
it possible to associate status codes with errors. This work leans heavily on
David Tolnay's excellent work on
anyhow
.
Under the hood http_types::Error
is a combination of a boxed error
+ status code. Just like anyhow
, any error can be cast to this type using
the ?
operator.
Additionally we also provide a ResultExt
trait that exposes a status
method on Result
that serves as a shorthand to cast to this type. This is
useful to quickly assign status codes to existing errors.
By default casting from a boxed error to http_types::Error
will assume it's a 500: Internal Server Error
. The .status
method is mostly useful for assigning other status codes.
/// Get the length of a file in bytes. /// Converts an `io::Error` to `http_types::Error`. async fn file_length(p: Path) -> http_types::Result<usize> { let b = fs::read(p).await.status(501)?; Ok(b.len()) }
This is a first step for us to make error handling a first-class citizen in HTTP frameworks such as Tide and Surf. We're still exploring how the general pattern feels, and would love to hear your thoughts!
Trait forwarding
But we're not using AsRef
exclusively for error handling; we use it
extensively throughout our libraries. This has been inspired by rustwasm's
approach to events in web-sys
. Rather than define a new trait that's shared
by objects, each DOM object implements
AsRef<EventTarget>
,
allowing it to be referenced as an EventTarget
.
In http-types
, we implement
AsRef
and
AsMut
to
convert between various types. For example a Request
is a combination of a
byte stream, headers, and a URL. So it implements AsRef<Url>
, AsRef<Headers>
, and AsyncRead
. Similarly a Response
is a combination of
a byte stream, headers, and a status code. So it implements AsRef<StatusCode>
, AsRef<Headers>
, and AsyncRead
.
Before adopting this pattern we played around for a bit with
defining custom traits, or following the pattern of exposing headers
/ headers_mut
in the interface. All of these felt rather awkward, and it was
only after playing around with
AsRawFd
more
that we realized AsRef
was the perfect fit here. And seeing it used in
practice in the web-sys
crates solidified that intuition further.
This pattern is repeated throughout our crates. Worth noting is that we don't
allow creating Headers
as a standalone type, yet it still serves as a
shared interface between Request
, Response
, and Trailers
. That means
that if you want to read or write to any of these types you can do:
fn set_cors(headers: impl AsMut<http_types::Headers>) { // set cors headers here } fn forwarded_for(headers: impl AsRef<http_types::Headers>) { // get the X-forwarded-for header }
And this code will happily take a Request
, Response
, or Trailers
and
perform the actions without a problem:
let fwd1 = forwarded_for(&req); let fwd2 = forwarded_for(&res); let fwd3 = forwarded_for(&trailers);
The plan is for Tide and Surf's Request
and Response
pairs to also
forward the same types. So whether you work with http-types
directly, or
targeting one of the frameworks the code you write to interface with them
remains the same.
Compatibility
Something else that we're excited about is the level of compatibility that we have with this new stack. It's now possible to run servers as Lambda functions , HTTP clients in the browser , and whichever combination of Rust server, TLS, DNS, and transport you want.
In order for http-types
to work with crates in the ecosystem that use hyperium/http
, we provide From
and Into
implementations for all our
types that can be enabled through a feature flag:
[dependencies] http-types = { version = "*", features = ["hyperium_http"] }
Because hyperium/http
doesn't provide a body implementation, compatibility
won't always be frictionless. But this is as close as we could get it, and
should cover many cases!
For us one of the most important cases to target is compatibility
with isahc
, an async HTTP library backed by curl.
Because it also uses AsyncRead
as its body, the compat layer makes working
with it really nice.
This means that between
http-client
,
http-service
, and http-types
we provide
an incredibly flexible layer that's widely compatible.
Design philosophy
Most of the work on these projects has been driven by curiosity. Trying to answer questions such as: "What does an HTTP library look like if it's designed for async Rust? Are generics required to allow TCP and TLS to be configured? Can HTTP/1.1 be expressed as streams?"
What we've tried to build is a set of libraries that will lower the barrier to experimenting with async HTTP in Rust. We use few new traits, limit the amount of generics we provide, compile incredibly fast, and closely follow Rust's naming conventions .
We feel the end-result is a set of libraries that are easy to pick up, easy
to hack on, and easy to maintain. async-h1
's implementation is
only a few hundred lines for the client and server each. Which means if you
ever hit a bug or want to make a change, it's possible to make the changes
without relying on resident experts. We hope this will empower people to try
things out, and continue to improve beyond where we've gone.
Acknowledgements
We'd like to take a moment to appreciate and thank the Hyperium team for all the work they've done. While we have concrete reasons to embark on a different path than the one they're on, the reality is that this has only been possible because of the work they've done.
To name examples: async-h1
makes use of the excellent
httparse
library. And
hyperium/http
has long been a center component of
Tide and Surf, and was the standard which we measured our work on http-types
against.
Even though the purpose of this post is to showcase the innovations we're introducing to HTTP in Rust, it's worth acknowledging that our groups have more in common than might first appear. We all want HTTP in Rust to succeed. We care about usability. We care about performance. The difference lies in how to achieve achieve those goals.
In the spirit of healthy competition, we think it's worth taking a moment to acknowledge our colleagues on the other side, and appreciate the work that they've done. Because we couldn't have done this without them. Thanks!
What's next?
We'd love for you to get involved! Come hang out with us on Discord , or consider helping out with documentation, tests, or issues on GitHub . Our main goals right now are to improve test coverage, iron out the bumps based on user feedback, and improve protocol support.
We've talked about authoring an async-h2
crate, but we currently don't have
the bandwidth to. We'd like to see Tide ship a 1.0 eventually, and continue
to improve protocol support and ergonomics to make Rust's HTTP stack not just
one of the fastest, but also the easiest to use of all languages.
Conclusion
In this post we've introduced three new HTTP libraries: async-h1
, async-native-tls
, and http-types
. These libraries have been built
natively for async byte streams, and provide a carefully balanced API that
combines ergonomics with performance.
It's hard to capture all the details that have gone into these libraries. Over the past 6 months we've been writing and rewriting these libraries over and over. We feel proud to be able to share them with you, and think it provides an exciting direction for HTTP in Rust.
We hope you'll enjoy the work we've shared today, and we're excited for you to try it out!
以上就是本文的全部内容,希望本文的内容对大家的学习或者工作能带来一定的帮助,也希望大家多多支持 码农网
猜你喜欢:本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们。
Ajax设计模式
Michael Mahemoff / 杨仁和 / 电子工业出版社 / 2007-5 / 78.00元
★本书荣获LinuxWorld Linux Journal2006年Editors' Choice awards。 ★绝好的一本ajax 高级读物,建议 每个web 程序员都需要买一本,了解什么是真正的好的web设计--网友 [精彩试读一] [精彩试读二] 本书是一本关于复杂Ajax应用的整体架构设计......一起来看看 《Ajax设计模式》 这本书的介绍吧!