Insufficient Error while Editing/Deleting Task in Salesforce

Recently I came across with an issue of getting in sufficient error (while editing or deleting task) by logged in user where he has created task and assigned to any other user without associating with any record. 

I found some facts during my research. I found that :  

If a user 'A' creates a task record and assign it to user 'B' then this task would not be visible to user 'A' until unless task is related to a record which is accessible to user 'A'. 

But even user 'A' can see that task through a way I just explained above, user 'A' would not be able to Edit or Delete that task record. 

So we may consider that through standard Interface ( standard layout/Home Page 'My Task' section or any report) it is not possible but I found that by accessing task records through Visualforce page we allow user 'A' to edit as well as delete task record created by him.