Based on the responses and replies being given being given it would appear as if the question meant to ask which day is the sabbath or holy day and not so much which day is the right day to worship on. We can worship on any given day! God is worthy to be praised on any day.
However, all references strongly evidence the 7th day sabbath. The sabbath is not only old testament seeing that Jesus himself kept the sabbath.
Luke 4:16 And he came to Nazareth, where he had been brought up: and, as his custom was, he went into the synagogue on the sabbath day, and stood up for to read.
If the sabbath was merely old testament then Jesus mission to show us how to keep the 10 commandment law would be nullified.
Matthew 5:17 ¶Think not that I am come to destroy the law, or the prophets: I am not come to destroy, but to fulfil. Some misinterpret this text to say that fulfil means to destroy. But think how foolish this would sound if Jesus said I came not to destroy the law but to destroy it. That's like saying I didn't come to this website to learn but to learn!
I'd like to also address Col 2:16.
Colossians 2:16 ¶Let no man therefore judge you in meat, or in drink, or in respect of an holyday, or of the new moon, or of the sabbath days. This was not refering to the 7th day sabbath. Notice the text says "sabbath days". In the old Israelite dispensation, there were many different kinds of sabbath days. This text was addressing those sabbath "days" and not the seventh day sabbath.
Hebrews 4 was not addressing an eternal rest. The text specifically said "day" which is singular.
4 For he spake in a certain place of the seventh day on this wise, And God did rest the seventh day from all his works.
5 And in this place again, If they shall enter into my rest.
6 Seeing therefore it remaineth that some must enter therein, and they to whom it was first preached entered not in because of unbelief:
7 Again, he limiteth a certain day, saying in David, To day, after so long a time; as it is said, To day if ye will hear his voice, harden not your hearts.
8 For if Jesus had given them rest, then would he not afterward have spoken of another day.
9 There remaineth therefore a rest to the people of God.
10 For he that is entered into his rest, he also hath ceased from his own works, as God did from his.
Some would have us believe that Sunday is the sabbath day. However, note that verse 8 says if he had spoken of another day then he would speak of it. Finally, understand that from an historical perspective, the Roman Catholic church under Pope Constantine in 321 A.D made the official change from Saturday to Sunday.
We can worship, give thanks and praise, pray and give allegiance to God on any given day. But as Exodus 20 points out , the 7th day is the sabbath of the Lord, thy God. In it thou shalt do no work, thou nor thy son nor they daughter....
Mark 2:27 points out quite clearly, And he said unto them, The sabbath was made for man, and not man for the sabbath:
The sabbath and a woman were given to man at creation. Read Genesis chapters 1 and 2 for the reference.
Some believe the sabbath was given to the Jews. Fine, then were women made for the Jews only? I think not. The sabbath is still binding and must be kept