Yes allspice.
It is a member of the myrtle family related to our crape myrtle.
The wood looks the same as well as the leaves.
They even have berries on them like the allspice tree does.
From everything I have read the crape myrtle isn't poisonous.
Nor its fruit.
Now I'm curious as to what the berries taste like.
What would the wood be like?
I do know it makes a fine walking stick.

You can look all over the place and they are not toxic and are sold as an off the beaten path spice.
Could it possibly be we have been letting an abundance of spice and wood just go to waste out of ignorance.
I think so.