Re:Apple: Transformer architecture optimized for Apple Silicon
by johnthuss on Thursday March 23, 2023 @11:25PM
(#35284714)
What prompt are you using for summarization? I’ve tried several variations without consistent results.
[ Parent
| on HN ]
Re:Apple: Transformer architecture optimized for Apple Silicon
by garblegarble on Friday March 24, 2023 @09:07AM
(#35288521)
On alpaca, I've found "Below is an instruction that describes a task. Write a response that appropriately completes the request. Summarise the following text: " or "Give me a 5 word summary of the following: " to work fairly well using the 30B weights.
It's certainly nowhere close to the quality of OpenAI summarisation, just better than what I previously had locally (e.g. in summarising a family history project with transcripts of old letters, gpt-3.5-turbo was able to accurately read between the lines summarising an original poem which I found amazing).
I half wonder if the change in spelling from US -> UK makes a difference...
I'd run a test on that but I've just broken my alpaca setup for longer prompts (switched to use mainline llama.cpp, which required a model conversion & some code changes, and it's no longer allocating enough memory)