To fluently know the fret board you must know it up and down. I would say you are thinking right. Learning up Sharps (#'s) and down Flats (b's) will be beneficial to progress.
The flat and sharps are like that because they was wrote and meant to played by standards.
Meaning just what they do. The note G forward from G would be like this
G# backwards from G
would be
Gb Keep in mind when you playing the major notes to watch for this, will help you to really find the right scale.
Also keep in mind of the
notes that do not have any sharps #'s or Flats b's.
They are B got no sharps #,C got no flats b,F got no flats b. By knowing this will help you a lot.
I do not know theory, but I remembered my mother saying this when she taught piano, many years ago.
But I think the formula have been changed for looks on the ledger.
I remembered seeing old songs wrote with b and # in the ledger at the same time.
But we also know, way way back in the days Blacks was self taught with help from God.