You don’t need to train anything if you just need embeddings. The data is text. You apply the pretrained model to your text and it returns the embedding. You save it in a vector database if you’re fancy, or a big numpy array if you’re like me. Then run your similarity search (cosine, Euclidean, etc).

a lot of embedding models have poor performance on domain specific data that is only mitigated with finetuning. alternately the instructor series mitigates this by fine-tuning the model on instructions and giving specific instructions to targeted domains.