revert: remove adapters / intrinsics / alora / lora from openai code#543
revert: remove adapters / intrinsics / alora / lora from openai code#543jakelorocco merged 6 commits intomainfrom
Conversation
|
The PR description has been updated. Please fill out the template for your PR to be reviewed. |
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
777d43d to
54262dc
Compare
|
Note: while reverting these changes, I found an issue with using format in generate_from_raw requests. Newer vllm server version expect different parameter structures. I added some code to try to find the vllm version and then use that version in the request. Added tests to test the underlying function. We don't have a great way to test different vllm server versions, but I tested older and newer versions manually. Both worked. |
|
@guicho271828, could you please review this when you get a chance. Thanks! |
52eb26b to
6848fd9
Compare
|
@guicho271828 context: the vLLM aLoRA PR will not accepted, so the alora/intrinsics code for openai is now all dead code. |
openai backend no longer supports intrinsics for vllm; vllm branch to support aloras was closed with prejudice.
dce4862 to
d6a9d62
Compare
0206743 to
065f7f3
Compare

Misc PR
Type of PR
Description
Currently removed all adapter based code;
will confirm if that's the intent or if we should just remove support for activate loras while still allowing for lora adapters.Consensus was to just remove all support and keep huggingface as the backend that supports adapters/aloras/intrinsics. The logic for this is that the vllm implementation that will eventually support aloras / intrinsics / arbitrary adapters will likely be a much different implementation. As a result, we are removing this code.Testing