Need help with context training

[guruprvn84]

We are evaluating NSQL to build our text-to-SQL model. As part of this exercise, we would like to train NSQL with our custom business-related datasets, definitions, and glossary. While finding the ways to do this we see LORA to be promising as it reduces the trainable parameters by 10,000 times. Though we are able to run LORA using the llama-2-7B model we are unable to find a way to train this model with custom datasets, definitions, and terms through the LORA technique. Could you please assist or point us to the right resources to do the same?

[senwu]

Our Numbers Station NSQL-llama-2-7B, is based on the llama-2-7B architecture, so any LORA fine-tuning done on llama-2-7B would apply to it. Please let us know if you have any specific issues or problems about LORA fine-tuning.

[sfc-gh-praj]

Hi @senwu ,

After fine tuning on custom dataset(using the example code provided in the repo), the model is not generating the expected output for the exact same query which in my training dataset.

Below query is part of my custom dataset

CREATE TABLE APPTABLE (
DATETIME DATE,
DURATION FLOAT,
RESPONSETIME VARCHAR() -- Using valid SQLite, answer the following questions for the tables provided above. -- sum of call duration week on week ?
)

My query which I have given the training dataset is
SELECT
TO_VARIANT(DATE_TRUNC('MONTH', DATE_TRUNC('WEEK', datetime)))::STRING AS month_start_date,
TO_VARIANT(EXTRACT(WEEK FROM datetime))::STRING AS week_number,
SUM(DURATION) AS monthly_weekly_call_duration
FROM
AppTable
GROUP BY
month_start_date,
week_number
ORDER BY
month_start_date, week_number

Now when I do the inference for the following question, its giving me the response which I not right:

text = """
CREATE TABLE APPTABLE (
DATETIME DATE,
DURATION FLOAT,
RESPONSETIME VARCHAR
) -- Using valid SQL, answer the following questions for the tables provided above. -- sum of call duration week on week ?

"""

model_input = tokenizer(text, return_tensors="pt").to("cuda")

generated_ids = model.generate(**model_input, max_new_tokens=100)
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))

Output --
CREATE TABLE APPTABLE (
DATETIME DATE,
DURATION FLOAT,
RESPONSETIME VARCHAR
) -- Using valid SQL, answer the following questions for the tables provided above. -- sum of call duration week on week ?

SELECT SUM(DURATION) FROM APPTABLE;

Output Generated is incorrect. Can you please share some inputs to get a better output as mentioned in my training dataset used for fine tuning.

Below is an example of the custom dataset which I have.

[senwu]

I understand that your training process did not go as planned, leading to a situation where the trained model is unable to memorize the training data. To monitor the training process, you can check the training loss regularly. In our experience, after fine-tuning, the loss can be close to zero. To fine-tune the model, you can adjust the learning rate, batch size, or gradient accumulation steps if you have it.

BTW:

CREATE TABLE APPTABLE (
DATETIME DATE,
DURATION FLOAT,
RESPONSETIME VARCHAR() -- Using valid SQLite, answer the following questions for the tables provided above. -- sum of call duration week on week ?
)

This example doesn't seem in the correct format and it doesn't match below.

text = """
CREATE TABLE APPTABLE (
DATETIME DATE,
DURATION FLOAT,
RESPONSETIME VARCHAR
) -- Using valid SQL, answer the following questions for the tables provided above. -- sum of call duration week on week ?

"""

model_input = tokenizer(text, return_tensors="pt").to("cuda")

generated_ids = model.generate(**model_input, max_new_tokens=100)
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))

[sfc-gh-praj]

I will try with adjusting the learning rate and gradient accumulation steps. Since I removed some unwanted column to keep the text short, I removed the "()" from the text VARCHAR() in the output which I have mentioned...But both are same..

Does spacing matters ?

[sfc-gh-praj]

HI @senwu : Can you please let me know if I can use the below sample data mentioned in the image for my fine tuning:

Please ignore the table name StreamliApp in the screenshot. It is AppTable and I have updated it.

I am testing the output using the below code after fine tuning. The below text is matching the instruction column value in the screenshot.

text = """
CREATE TABLE APPTABLE (
DATETIME DATE,
DURATION FLOAT,
RESPONSETIME VARCHAR(16777216)) -- Using valid SQLite, answer the following questions for the tables provided above. -- sum of call duration week on week ?

"""

model_input = tokenizer(text, return_tensors="pt").to("cuda")

generated_ids = model.generate(**model_input, max_new_tokens=100)
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))

I want to ensure I am using the dataset for fine tuning in the required format. Also my prompt matches to that for the data in the training set which I have used for fine tuning.

Your inputs will be of great help.

[senwu]

We use the following prompt format:

Input:

CREATE TABLE stadium (
    stadium_id number,
    location text,
    name text,
    capacity number,
    highest number,
    lowest number,
    average number
)

CREATE TABLE singer (
    singer_id number,
    name text,
    country text,
    song_name text,
    song_release_year text,
    age number,
    is_male others
)

CREATE TABLE concert (
    concert_id number,
    concert_name text,
    theme text,
    stadium_id text,
    year text
)

CREATE TABLE singer_in_concert (
    concert_id number,
    singer_id text
)

-- Using valid SQLite, answer the following questions for the tables provided above.

-- What is the maximum, the average, and the minimum capacity of stadiums ?

Ouput is the corresponding SQL to answer the question.

Also you can find more examples here: https://huggingface.co/datasets/NumbersStation/NSText2SQL