Your Ledger device must stay plugged in and running the baking app to be able to bake and endorse blocks. Once the device is set up, you do not need to manually sign when you have rights - it will automatically sign for you!
It sounds like you have already imported the address you'd like to bake with. If not, plug your Ledger into your machine and open the Tezos Baking app. Then run:
tezos-client list connected ledgers
You should see that it found your device and provides a few commands for importing addresses. Here's an example of one of those which imports an address from the device and gives it the alias
ledger_alias, which you'll use to refer to your address when running other commands:
tezos-client import secret key ledger_alias "ledger://adjective-animal-adjective-animal/ed25519/0'/0'"
Once imported, we can 1) setup the Ledger to bake for
ledger_alias and 2) register
ledger_alias as a delegate (baker) on the blockchain.
tezos-client setup ledger to bake for ledger_alias
If your Ledger isn't authorized, the device will read 'Baking Key: No Key Authorized' from the baking app. Once it is authorized, it will show your PKH (most likely a
tezos-client register key ledger_alias as delegate
Then start your baker and endorser so they use that account:
tezos-baker-003-PsddFKi3 run with local node ~/.tezos-node ledger_alias
tezos-endorser-003-PsddFKi3 run ledger_alias
That's it! For more information, here's the full docs for Tezos Baking - https://github.com/obsidiansystems/ledger-app-tezos#using-the-tezos-baking-application