Why does the client-side middleware use Arc
If you’ve paid close attention, you’d notice that in the generated code on the client-side, we wrap the user-provided
Arc<Req> before actually executing the volo_thrift client’s
However, on the server-side, we directly use
The reason for this design is that the client-side requires more complex service governance logic compared to the server-side.
Especially, some service governance logic conflicts with Rust’s ownership model. For instance, if a connection fails, we might need to retry on a different node or even implement more complex timeout-retry logic.
If we were to directly use
Req, once we execute the inner service’s
call for the first time, the ownership would have already been moved, preventing us from implementing retry logic.
Arc helps us avoid problems caused by concurrent access under middleware (such as scenarios involving mirror/diff), without going into excessive detail here.
Furthermore, the client-side itself shouldn’t modify
Req, hence eliminating the need for mutable access.
On the other hand, the server-side doesn’t have such complex usage scenarios, and ultimately, ownership needs to be passed to the user’s handler. Therefore, using
Req ownership directly on the server-side suffices.
Why is the code generated by volo-cli separately split into the volo-gen crate?
This separation is because Rust’s compilation operates on a crate-by-crate basis. Creating the generated code as a separate crate allows for better utilization of the compile cache (IDL generally doesn’t change frequently).
How compatible is it with Kitex?
Volo is fully compatible with Kitex, including functionalities like metadata transmission.
Where did poll_ready (backpressure) go?
In Tower’s Service, there is a method called
poll_ready, used to ensure downstream Services have sufficient processing capacity before making requests and to provide backpressure when processing capacity is insufficient.
This is a very ingenious design, and Tower elaborates on the reasons for this design in the article inventing-the-service-trait.
However, based on our real development experience, we have summarized the following insights:
- The majority of
poll_readyimplementations directly call
self.inner.poll_ready(cx); the remaining implementations are even simpler, directly returning
poll_readygenerally does not actually check the load across services (i.e., it does not send a request downstream asking, “Hey buddy, can you handle more?"), so it usually involves evaluating certain conditions within local middleware (such as Tower’s example of rate limiting middleware).
- Based on the above two points, almost all
poll_readyscenarios can achieve the same effect directly within the
callmethod. In practice, the outer layer of the
servicesimply waits when returning
Poll::Pending, so it is more ergonomic to write code using
- As for the potential issue of resource wastage, intercepting middleware is generally best placed as early as possible. Therefore, by properly arranging the order of middleware, this issue can be resolved.
Therefore, following the principle of “if it isn’t necessary, don’t include it” and to enhance usability, we have ultimately decided not to include the
poll_ready method in our design.
Was this page helpful?