There is some evidence of physical Hebrew Bible manuscripts written well before the New Testament. Radiocarbon dating and the handwriting style of various Dead Sea scrolls (Isaiah, Psalms, Deuteronomy, etc.) places them between 300–100 BCE. The Great Isaiah Scroll is often dated around 150–100 BCE.
As far as the New Testament, Jewish communities in Egypt translated the Hebrew Bible into Greek in the Hellenistic period. Surviving Greek papyrus copies from before the 1st century CE include:
the Rylands Papyrus 458 (Deuteronomy), roughly 2nd century BCE
the Papyrus Fouad 266 (Genesis/Deuteronomy) about 1st century BCE
the Nahal Hever Greek “Minor Prophets” roughly 50 BCE–50 CE. The Letter of Aristeas (2nd century BCE) mentions the translation, showing that it was known and used.
It's true that many religious relics have disappeared, but as you say, the texts do remain. Which texts remain in their original authored form is debatable.
To some Christians, it doesn't matter that the Bible (various scriptural canons by branch) is their only inheritance, because to the ones who value the Gospel and the teachings of Jesus, there's no greater treasure in the world.
Proverbs 2:4–5
“If thou seekest her as silver, and searchest for her as for hid treasures; Then shalt thou understand the fear of the Lord, and find the knowledge of God.”
Matthew 13:44–46
“Again, the kingdom of heaven is like unto treasure hid in a field; the which when a man hath found, he hideth, and for joy thereof goeth and selleth all that he hath, and buyeth that field. Again, the kingdom of heaven is like unto a merchant man, seeking goodly pearls: Who, when he had found one pearl of great price, went and sold all that he had, and bought it.”
If you think you’ve underrated it and you want to know what you're missing, read it. You don't need to be Buddhist to read the suttas, or Taoist to read the Dao de Jing, or Christian to read the Bible. There's a lot to be inspired by even if you don't subscribe to everything presented.
When I first took an interest, I moved through the entire thing chronologically, on audiobook, at 1.5-2x speed. I quickly got the broad strokes and picked up details I wouldn't have if I weren't listening to it. David Suchet has audio recordings on YouTube that are quite nice. Knowing the genre of what you're reading is also helpful for engaging with it (ie narrative, historical accounts, poetry, etc).
Once you have a mental framework for the Tanakh / Old Testament / New Testament, it becomes easier to engage in apocryphal literature of both Jewish and Christian tradition. That helps to develop a fuller picture of the traditions outside of the mainstream presentation, and you'll begin to see where threads connect to non-Abrahamic lineages.