[Prompt Tuning] Multi-task Prompt Tuning

Multi-task Prompt Tuning

Multitask Prompt Tuning enables Parameter-efficient Transfer Learning

https://openreview.net/pdf?id=Nk2pDtuhTq 

 

MPT를 활용하면 여러 작업에 적용되는 단일 소프트 프롬프트를 만들 수 있습니다.

 

 

MPT is a variant of prompt tuning that leverages a transferable shared prompt that has distilled knowledge from multiple tasks. The shared prompt is then tuned with the user’s data. 

This technique was developed in the MIT-IBM Watson AI Lab.

 

Empirical results demonstrate that MPT outperforms prompt tuning across a number of benchmarks.

Another MPT differentiator is the ability to tune a single prompt on several tasks.

 

MPT는 여러 작업에서 얻은 지식을 추출한 전송 가능한 공유 프롬프트를 활용하는 프롬프트 조정의 변형입니다. 그러면 공유 프롬프트가 사용자 데이터에 맞게 조정됩니다.
이 기술은 MIT-IBM Watson AI Lab에서 개발되었습니다.

경험적 결과에 따르면 MPT는 여러 벤치마크에서 즉각적인 조정보다 성능이 뛰어납니다.
MPT의 또 다른 차별화 요소는 여러 작업에 대해 단일 프롬프트를 조정하는 기능입니다.

---

References

• https://github.com/huggingface/peft
• https://heidloff.net/article/introduction-multi-task-prompt-tuning/