add dedicated prompt tracking

[pritamsoni-hsr]

No description provided.

[pritamsoni-hsr]

@akhatre , this can be util provided in sdk, which allows us getting most accurate prompt template

[akhatre]

@pritamsoni-hsr so

• span level -> we store formatted prompt as input, template as attribute. Should we also store variables?
• prompt model -> only stores templates. Should we have some logic to add templates from attributes of spans when provided by the user?

besides the above, we should have a way to link spans to prompt model. Because:

• For prompt tuning we need to select N spans by their template
• For cases when we auto-extract templates from traces we would want to store that template and link it to traces it originated from
• For backtesting new prompt templates we need both the template, the spans and their variables to create a new formatted prompt to test on the backend

[pritamsoni-hsr]

gonna further speak with rohit on this, what would be the best to consume